Effects of tiered training on general educators' use of specific praise.
Article Type:
Report
Subject:
Teachers (Training)
Authors:
Thompson, Michele Terry
Marchant, Michelle
Anderson, Darlene
Prater, Mary Anne
Gibb, Gordon
Pub Date:
11/01/2012
Publication:
Name: Education & Treatment of Children Publisher: West Virginia University Press, University of West Virginia Audience: Professional Format: Magazine/Journal Subject: Education; Family and marriage; Social sciences Copyright: COPYRIGHT 2012 West Virginia University Press, University of West Virginia ISSN: 0748-8491
Issue:
Date: Nov, 2012 Source Volume: 35 Source Issue: 4
Topic:
Event Code: 280 Personnel administration
Organization:
Government Agency: United States. Department of Education

Accession Number:
310741888
Full Text:
Abstract

Research suggests a compelling correlation between teacher behavior and effective learning environments. Focusing on the evidence-based teaching skill of offering behavior-specific praise (BSP), the researchers worked with three elementary-level general educators in a tiered model of training generally known as response to intervention (RtI). Although RtI commonly provides targeted instructional support to students, this study used the RtI framework to provide professional development instruction to teachers. The researchers also tracked the behavior of three students identified by the teachers as having behavioral difficulties, who became the focus of each teacher's BSP. Results showed increases in rates of BSP following the Tier 2 and Tier 3 interventions (video self-monitoring and peer coaching), but not following the Tier 1 intervention (school-wide in-service training). Averages for all three students' on-task behavior increased with increased teacher BSP.

KEYWORDS: behavior-specific praise, response to intervention, faculty peer coaching, video self-monitoring, professional development, tiered training

Improving public education is of national concern as many schools grapple with low achievement results in the context of legislative mandates for increased student achievement and highly qualified teachers (No Child Left Behind Act, 2001). The ability of a teacher to manage student behavior has been emphasized as a skill that leads to increased learning time and improved academic and social outcomes (Simonsen, Fairbanks, Briesch, Myers, & Sugai, 2008). In particular, the use of behavior-specific, contingent praise has been documented as a teaching practice that consistently results in improved student academic and social behavior (Cherne, 2009; Sugai, 2007). However, significant evidence indicates that teachers rarely use praise effectively in the classroom (Beaman & Wheldall, 2000; Brophy, 1981; Burnett, 2002; Ferguson & Houghton, 1992; Sutherland, Wehby, & Copeland, 2000). This study explored a tiered professional development structure that was used to teach teachers to use behavior-specific contingent praise in dealing with disruptive student behavior.

Background and Literature Review

Creating professional development systems that effectively support and sustain teachers' use of identified effective practices can be difficult (Guskey & Yoon, 2009). Several forms and strategies are common at present: meetings and workshops, self-monitoring, and instructional coaching. These will be discussed in detail, below.

The most typical professional development strategy includes meetings or workshops in which participants passively listen to didactic instruction. Research suggests several drawbacks to this type of teacher training (Sprick, Knight, Reinke, & McKale, 2006). First, little to no follow-up training or implementation accountability occurs. Second, passive delivery gives attendees few opportunities to practice for skill mastery. Finally, and perhaps most important, little evidence of generalization to classroom implementation exists (Elmore, 2002; Garet, Porter, Desimore, Birmon, & Yoon, 2001; Garet, Wayne et al., 2010; Fixsen, Naoom, Blase, Friedmand, & Wallace, 2005; Myers, Simonsen, & Sugai, 2011; Yoon, Duncan, Lee, Scarloss, & Shapley, 2007).

Self-monitoring is a professional development strategy that provides teachers with data on which to reflect, making it effective for changing a variety of behaviors in various settings (Kalis, Vannest, & Parker, 2007). Kalis et al. had teachers self-monitor using a pocket counter, which they clicked to record instances of behavior-specific praise, with time allotted for analyzing the data. This simple cost-effective method makes the teacher aware of his or her use of a targeted skill, but teachers must be able and committed to accurately collect data during instruction.

Teachers can also self-monitor by video recording their lessons, thus collecting data for evaluating self and student behaviors without interrupting the flow of lesson delivery (Sherin & van Es, 2005) and creating a permanent product that may decrease inaccuracy in data collection. These self-monitoring tools offer reliable measures of teacher behavior and enable an efficacious follow-up procedure that has been shown to increase the likelihood of treatment implementation (Noell et al., 2005).

Autonomous performance feedback can be as simple as creating a graph of collected data, listening to audio recordings, or viewing video. However, self-monitoring tactics, when employed without involvement from an experienced peer such as a skilled instructional coach, can be ineffective, confusing, and impractical to teachers, leaving them without a clear path to positive change (Colvin, Flannery, Sugai, & Monegan, 2009; Joyce & Showers, 1995; Sprick et al., 2006).

In addition to workshops and self-monitoring tools, instructional coaching can be effective in teacher professional development (On-chwari & Keengwe, 2008; Stichter, Lewis, Richter, Johnson, & Bradley, 2006). Instructional coaching addresses the needs identified by teachers to tackle specific individual concerns, learn in collaborative professional environments, and receive ongoing support by competent peers (Guskey & Yoon, 2009).

Coaching is an intensive intervention requiring a positive working relationship with a colleague. Spontaneous coaching or consultation in natural settings can be as effective as more formalized coaching structures. Research relating coaching to student outcomes is inconclusive (Garet, Porter et al., 2001); however some studies suggest improvements in teacher ability and confidence (e.g., Sprick et al., 2006).

Studies of coaching models and efficacy commonly suggest a need for structure to the coaching process if it is to produce change (Peterson, Taylor, Burnham, & Schock 2009; Sprick et al., 2006; Stichter, Lewis, Richter, Johnson, & Bradley, 2006). Components include (1) school-wide common classroom management practices, (2) observational guides, (3) pre-conferences to determine target teaching skills, (4) post-conferences to collaboratively analyze direct observation data, (5) intervention choices (such as modeling or observation in other classrooms), (6) goal setting and follow-up, and (7) repetition of the process as needed.

These three professional development strategies, meetings and workshops, self-monitoring, and instructional coaching, can be incorporated as tiered level support for teacher improvement. When the three are used sequentially--with each increasing in intensity in response to perceived need--the approach is similar to response to intervention (RtI). Generally used for students, RtI is a multi-tiered problem-solving approach used to proactively apply high quality evidence-based learning strategies matched to student need according to data (Ardoin, 2006; Barnett, Daly, Jones, & Lentz, 2004; Fuchs & Fuchs, 2006; Gresham, 2005). Numerous studies have been conducted using Rd to support students' academic and social behavior at school, yet limited research has examined the use of RtI in professional development for teachers (Coyne, Kame'enui, & Carnine, 2007; Kame'enui, 2007; Myers et al., 2011).

A study by Myers, Simonsen, & Sugai (2011) applied an RtI approach to enhance teacher behavior. Teacher participants were self-nominated general and special educators in a middle school who had contacted the researcher seeking assistance with excessive and disruptive student behavior. The study took place in schools that were successfully implementing school-wide positive behavior support; thus all staff had received training to support positive classroom behavior--considered the universal or primary intervention.

Myers et al. (2011) defined as essential criteria a ratio of four positive to one negative interaction with students and six praise statements per 15-minute observation. After training had been presented on behavior specific praise (BSP) rates and positive to negative interaction ratios, a teacher evaluation was conducted to determine acquisition of these skills. Four of the self-nominated teachers who had not responded as desiring the training became the participants in the study. These four received a secondary or more intensive intervention, which included meeting weekly with the researcher who (a) provided visual feedback in the form of a graph showing BSP rates, positive to negative ratios, and student on-task behavior, (b) praised improvements in teacher and student outcomes, (c) offered recommendations for change, and (d) identified goals with the teacher for the next observation. If criteria were not met at this level, a tertiary intervention was introduced, which included a more intensive feedback schedule (following each observation), additional suggestions for increasing praise rates and positive to negative ratios, and more individualized support. Some of the noted limitations of the Myers study are: (a) students were selected randomly by the trained observers versus by way of a pre-determined selection criteria, (b) teachers were self-nominated, and (c) the schools were participating in school-wide positive behavior support training and implementation.

This current study is a systematic replication of the Myers et al. (2011) study. The limitations noted above informed the design of the current study described in this manuscript. Differences are as follows: (a) general education teachers in elementary school settings were the targeted participants, selected from a pool of principal-nominated teachers, (b) participating schools were not involved in or monitored for school-wide positive behavioral support training, (d) intervention included behavior specific praise training at the universal level, video self-monitoring at the secondary level, and coaching at the tertiary level, (e) data were collected on teacher-delivered BSP and student on-task behavior, and (f) criteria for teacher praise rates (PR) were determined using baseline PR with a percentage increase as opposed to a pre-determined number of praises per minute. Table 1 indicates differences as compared with the Myers et al. study.

This study adds to the literature on effective professional development methods by examining the effects of tiered interventions on teacher behavior. The primary purpose of this study was to specifically evaluate the relationship between an RtI approach to teacher training and the frequency with which general education teachers implemented behavior-specific praise. This study was not designed to explore the relationship between teachers' delivery of BSPs and student behavior; however, the researchers did collect supplemental evidence of teachers' BSPs corresponding effects on student behavior. Two ancillary purposes of the study were to explore (a) the educators' perceptions of the utility and effectiveness of the intervention, and (b) the effects of elementary general educators' behavior-specific praise on the on-task behaviors of students identified as being disruptive.

Method

Participants

Selection process. The first author met with the principals of four elementary schools to discuss the purpose of the study and request names of three to four general education teachers per school who might participate. The nominations were based on concerns for unresolved disruptive student behavior and/or specific requests of teachers for additional behavior management support. Twelve teachers (three in each of the four schools) were contacted by the first author and by their principal and informed that behavior data would be collected in their classroom in an effort to provide behavioral support and selection purposes. Data were collected on the frequency of all twelve nominated teachers' BSP. Following this series of data, three participant teachers were identified, and ultimately chosen, based on three criteria: (a) principal nomination of a teacher who indicated that she had one student with disruptive behavior, (b) frequency of BSP less than one per 5-minute interval as observed by a district intervention team paraeducator over several 15-minute observations, and (c) agreement to participate in the study, as indicated by signing a consent form.

The criteria for student participation were that the student (1) did not at that time have a formal behavior intervention plan and (2) would be present for the majority of the observation period. One was chosen from the class of each teacher; consent from parents and assent from students were obtained.

Teacher participants. Three white female elementary teachers-"Anna," "Jane," and "Gail,"--participated in the study. All were between ages 40 and 50. Two had earned bachelor's degrees in education and accumulated over 10 years of classroom teaching experience, and one had previously taught art as a classified employee and at the time of the study was working towards certification in her first year of an alternative licensure program (ARL). None of these teachers were trained formally in Positive Behavior Support or RtI methods. Their training in behavior and classroom management is minimal, deriving only from their teacher preparation programs. (See Table 2.)

Student participants. Student participants were three Caucasian males, ages 8, 10, and 11. All three were reported by their teacher to be noncompliant and disruptive in class; one had an individualized education plan (IEP).

The coach. The coach, the primary researcher for this study, was a female certified special educator with a BA degree and 10 years of teaching experience. She was working as a program specialist for the district special education department.

Settings

The study took place in three public elementary schools of a suburban district in the Western United States.

School A. School A had a total student population of 691, with 22.1% qualifying for free and reduced price lunch, 1.6% learning English as a second language, and 13.5% receiving special education services. The study was conducted in a general education class of 31 fourth-grade students, including two who had IEPs and two who showed attention difficulties. The classroom management system included a "three strikes" approach: After three reminders to comply, consequences followed each offense. Researchers noted a relaxed atmosphere with student-teacher relationships that were more familiar than formal.

School B. School B had a total student population of 535, with 60.7% receiving free or reduced price lunch, 13.5% learning English as a second language, and 18.5% qualifying for special education services. The study was conducted in a general education class of 26 second-grade students, six of whom were designated by the teacher as having attention and behavior difficulties. The classroom behavior management strategy was a chart with colored cards: green for acceptable behavior, yellow for failure to follow instructions despite reprimands/corrections, red for teacher conference and parent contact. Students with yellow or red cards lost certain privileges. The teacher interacted with her students in a familiar manner and delivered consequences directly.

School C. School C had a student population of 840, with 26.7% receiving free and reduced price lunch, 3.5% learning English as a second language, and 13.1% receiving special education services. The study took place in a general education class of 26 fourth-grade students. The teacher indicated that five students had IEPs and four students had attention issues or non-compliant behaviors. A token economy system was used, with each student receiving a ticket at the beginning of the class with opportunities to earn additional tickets throughout the day; later the tickets could "buy" items. The teacher was approachable yet more formal than familiar with her students.

Dependent Variables

Teacher behavior. The general education teacher behavior recorded as the dependent variable in this study was the frequency of behavior-specific praise statements for students' academic and social conduct. Behavior-specific praise (BSP) is defined as a verbal statement (a) indicating approval, (b) describing a specific desired social or academic behavior exhibited by the student, and (c) including a praise word (e.g., great, appreciate, excellent).

* "Sam, I appreciate the way you asked James to join you in the group activity."

* "Jane, you did a great job helping Megan figure out that problem."

* "Troy, you defined that vocabulary word so well. Now you will be able to understand the story!"

Vague positive statements not linked to specific behavior ("Great job!" "Super!" "Good!") were not acceptable.

Student behavior. On-and off-task student behaviors were observed and recorded in conjunction with teacher behavior. On-task behavior included students orienting themselves towards the teacher: for example, taking notes on teacher lectures, raising a hand to ask a clarification question, or performing tasks when directed by the teacher. Student off-task behavior was defined as the student not orienting to the task or work when directed by the teacher. Such behaviors might include a student looking in his desk or out the window, talking with a peer about a nonrelated subject, putting his head on the desk, or doing an unrelated activity. Also considered off task was disruptive behavior, defined as behavior that distracts the flow of instruction and the learning of other students (e.g., shouting, talking to other students during teacher instructions, making noises, or throwing objects).

Independent Variable

An RtI framework was implemented to support teachers in learning and implementing behavior strategies. Once the teachers were taught the strategies, their use of them was evaluated. Teachers advanced from tier to tier based on their progress, or the lack of progress, within the tiered framework of predetermined criteria.

Tier 1--School-wide training in behavior-specific praise. The Tier 1 (primary) intervention was a one-time training session held during a 30-minute faculty meeting. The researcher conducted this presentation, which (a) defined general and behavior-specific contingent praise for social and academic student behavior, (b) shared research on the effectiveness of using high rates of BSP to increase students' positive behavior, and (c) provided teachers opportunities to practice verbalizing BSP statements. At the conclusion of the training, all were encouraged to increase current personal BSP rates by 50%. This criterion was selected by the primary author as a manageable goal for most teachers to attain when asked to increase their frequency of praise. Instead of comparing them to an unknown, ambiguous standard, as the literature is unclear what the standard should be for praise rate, the author chose to encourage the participants to use their own baseline as the standard from which to improve.

Tier 2--Video self-monitoring. Participants at the Tier 2 level video recorded themselves teaching a lesson segment of at least 15 minutes but no longer than 25 minutes. While watching the video, they self-scored the data on BSP rates by counting the total number of BSPs during a 15-minute teaching segment and sent the numerical data to the experimenter via email.

Tier 3--Coaching. A coach, who was the first author, was implemented at Tier 3 to provide non-evaluative support and to guide the teacher through the problem-solving cycle. The coach sent emails giving specific praise for data collection and improved rates of BSP and also made personal visits providing encouragement and sharing/ discussing data. Additionally, a variety of interventions were offered to the participants, including use of a MotivAider, a device that vibrates to signal fixed/intermittent time intervals, (Behavioral Dynamics, 2010), continued use of the Kodak FLIP video camera, and opportunities to observe in other classrooms or to have the coach teach a lesson segment in the participant's classroom demonstrating a high frequency of BSP.

All participants chose to use the MotivAider (Behavioral Dynamics, 2010) to prompt delivery of BSP to the target student. Participants were encouraged to achieve a 50% increase in the average frequency of BSP, individualized goals being calculated by multiplying the frequency of praise of previous intervention conditions by 1.5.

Data Collection Procedures

Measurement. Data were collected on the dependent variables by direct observation of both teachers and students. Teacher behavior (BSP) was recorded using event recording with paper and pencil during a standard 5-minute observation session. The researchers then calculated and graphed the frequency with which BSP was delivered by each teacher per observation session.

Student behavior was recorded by a paper/pencil momentary time sampling at the end of each 10-second interval: a "+" sign for an on-task interval and a "-" sign for off-task behavior. As the researchers primarily focused on increasing specific teacher behavior, the momentary time sampling was sufficient to determine the potential impact on student behavior. Data were monitored daily by the primary researcher through emails to the observers and participants. Raw data were recorded on an Excel spreadsheet, which created a numerical sequential list and also generated a line graph.

Observers and observer training. Data were collected by the primary researcher and two paraeducators from a district intervention team, who had 5 to 15 years of training in collecting teacher and student behavior data. Both paraeducators were white females, one age 51 and the other age 40. Both spent most of their working day in general education classrooms providing support as needed for students with various learning and behavioral disabilities.

The primary researcher provided weekly training to the observers in data collection procedures for this particular study. Training included mastering behavioral definitions, distinguishing between examples and non-examples of written and video examples of behavior, and practicing recording data. Trainees were required to obtain 100% accuracy.

Interobserver agreement (IOA) data were calculated on 31% of the sessions across all experimental conditions. Data were compared, and agreement was defined as two independent observers marking the same total number of tallies for teacher behavior "+" and marks for student behaviors. For teacher behavior (event recorded behavior) agreement was calculated by dividing the lower total by the higher total x 100. For student behavior (interval recorded behavior), agreement was calculated using the formula: number of agreements divided by the number of agreements plus disagreements X 100. If IOA dropped below 85%, the observers were retrained using the training steps previously outlined, which only happened twice. Average IOA for BSP was 100%, with a mean of 100% for both baseline and intervention. Average IOA for student on-task behavior was 95% (range 81-98%), with a mean of 95% for both baseline and intervention. The overall IOA for teacher and student behaviors was 97.5%.

Experimental Design

This study used a multiple probe design across participants to evaluate the effects of the independent variable.

Baseline. During baseline conditions, with no systematic professional development addressing BSP, the frequency of teacher BSP and student on-task behavior data were simultaneously collected by the observers according to standard intervention team procedures. Data were collected when the teacher was engaged in a 15-minute direct teaching segment, at a specific time each day. If frequency of BSP was < 2 per 15-minute observation period, the school was selected to continue with the study, including the school-wide faculty training on BSP.

Tiers of intervention. Intervention was structured in three tiers, comparable to those used in the response to intervention (Rtl) approach.

Tier 1. The school-wide faculty training was the first condition (primary tier) of intervention. Collection of data on teacher BSP and student on-task behavior continued for all 12 principal-nominated teachers at least three times per week by observer paraeducators assigned to that classroom. As was mentioned participant selection section, these data were collected for the purpose of providing behavioral support and selecting participants. One teacher from each of the three schools was selected as a participant for the study based on her frequency of BSP, availability, and willingness to participate for the duration of the study. A participant whose frequency of BSP was greater than or equal to a 50% increase from baseline would remain at Tier 1 intervention; those who were below a 50% increase from baseline moved to the Tier 2 intervention.

Tier 2. Tier 2 intervention conditions added a self-monitoring process, whereby participants would use a Kodak FLIP video camera to tape 15-minute lesson segments of their own teaching. Participants collected BSP data from watching the video, recorded the total BSP counts during a 15-minute segment, and sent the data to the primary researcher via email at least three times per week. The researcher kept the participant data while the intervention team continued to collect data on teacher BSP and student on-task behavior. When the frequency of BSP observed by the research team were greater than or equal to a 50% increase from Tier 1, the participant continued to use video self-monitoring until at least three consecutive data points indicated an increase of at least 50% from baseline. If the frequency of BSP from the observer fell below a 50% increase from Tier 1 for two or more data points, the participant moved to Tier 3.

Tier 3. Tier 3 intervention continued the video self-monitoring process and added a coach--the primary researcher. She examined the graphed data with the participant and asked the following reflective questions: What did you observe during video self-monitoring? What did you notice about your data? What strategies for increasing BSP are effective for you? What have you noticed about student behavior?

Observers continued to collect data. When the frequency of BSP were greater than or equal to a 50% increase from Tier 2, coaching was minimized to include only the researchers' encouragement and behavior-specific praise to the participant at least three times per week. If BSP from the observer dropped below a 50% increase from Tier 2, the coach reintroduced video self-monitoring and increased personal visits until three consecutive data points showed an increase in BSP that was greater than or equal to a 50% increase from Tier 2.

Treatment Fidelity

To ensure proper treatment implementation, a checklist was developed for each condition of the study. Treatment fidelity was calculated as the total number of steps followed, divided by the total number of listed steps x 100. Data on treatment fidelity are reported below.

Tier 1. Treatment fidelity at Tier 1 included the researcher's use of a lesson plan to guide the delivery of instructional content during faculty training and a checklist outlining the steps of the Tier 1 procedure. The checklist was marked by the researcher prior to and following the training and was integral to the Tier 1 data collection process. As the sole provider of teacher training at Tier 1, the researcher self-evaluated the fidelity of the treatment at 100%. No reliability measure was included at this stage of the study.

Tier 2. Treatment fidelity during Tier 2 was ensured through participants' use of a checklist to monitor their implementation of intervention procedures: (a) operating a Kodak Flip camera to record a 15-minute or longer teaching segment, (b) viewing the video recording and tallying the BSP, (c) totaling the BSP and emailing the data to the researcher the same day, and (d) implementing the video process at least three times per week. The researcher independently recorded on the checklist her responses to the teacher emails and the written praise she provided. The primary researcher also made unscheduled random visits to check the video camera and watched the recorded contents to ensure recordings were taking place. Although reliability data were not formally collected, the researcher's written records indicated the average percentage of intervention steps completed by teachers was 88%.

Tier 3. Treatment fidelity at this level consisted of the researcher recording personal participant visits on a coaching log. Participants were asked to continue the video self-monitoring, which included sending BSP data to the researcher via email. The permanent product of the recorded teaching session served as an additional treatment fidelity check. The average percentage of steps completed across all three interventions, including participant and researcher responsibilities, was 92%.

Social Validity

Each participant completed a post-intervention questionnaire at the conclusion of the study to evaluate perceptions about the utility, effectiveness, and practicality of a tiered framework for professional development and the use of BSP to manage disruptive students. All participants received, filled out, and returned the questionnaire electronically, rating 10 items on a 6-point Likert-type scale. Eight questions had rating choices from strongly disagree to strongly agree. The scale for the remaining two questions included almost never, almost always, and not applicable as the options. Two participants also completed a section inviting comments. The researcher encouraged them to be candid in their responses. The results are listed in Table 3.

Results

The overall results indicate limited change from baseline to the Tier 1 (faculty training) intervention on increasing BSP, with teacher behavior change increasing at the Tier 2 (video self-monitoring) and Tier 3 (coaching) intervention levels (see Figure 1). Concurrently, student on-task behavior, although highly variable, showed an increasing trend as teachers increased their BSP.

Participants' Behavior During Study Phases

Anna.

Baseline. Prior to Tier 1 intervention Anna gave no behavior-specific praise across three observations--a zero trend with low, stable data. The student participant averaged 82% time on task during baseline--a high level with moderate variability and a trend for slight increase.

Tier 1. Following Tier 1 intervention Anna's average frequency of BSP was 0.2, and student on-task behavior averaged 64% with a range of 40% to 78%. With BSP at 0 prior to Tier 1, Anna needed to increase BSP to 1 or more in over three consecutive data points to calculate a 50% increase. The criterion was not met to remain at Tier 1; Anna moved to Tier 2 intervention.

Tier 2. BSP during Tier 2 averaged 1.1 per 5-min observation and ranged from 0 to 5. Student on-task behavior averaged 61% with a range of 38% to 79%. Although Anna's BSP increased from 0.2, she had consecutive data points with 0 BSP and thus was moved to Tier 3 intervention.

Tier 3. Anna averaged 2.6 BSP with a range of 0 to 7 across 12 observations during Tier 3 intervention. Student behavior averaged 68% time on-task with a range of 38% to 89%. The corresponding BSP for the lowest student on-task percentage was 0; the day of the highest student on-task behavior percentage, the frequency of BSP was 3.

Gail.

Baseline. Prior to intervention Gail had no BSP over seven data collection points. The student's on-task behavior averaged 44%.

Tier 1. After Tier 1 intervention Gail gave no BSP five out of seven days, averaging 0.3 BSP per 15 minutes for Tier 1 condition. Due to consistent data points with no BSP, she was moved to Tier 2. The student's average time on task was 41% with a range of 17% to 67%.

Tier 2. During Tier 2 Gail's average frequency of BSP was 8.6 over 11 observations with a range of 3 to 13 BSP per 5-minute observation. Student on-task behavior averaged 62% and a range of 14% to 91%. Because Gail consistently maintained a frequency of BSP above the 50% improvement over Tier 1 rates at a high stable level with a rapidly increasing trend, she remained at Tier 2 and faded use of the video camera for the last three observation periods.

Jane.

Baseline. Prior to the faculty training on BSP, Jane's average frequency of BSP was .44 per 5 minutes over nine observations. Average student time on-task was 36% with a range of 2% to 74%.

Tier 1. After Tier 1 intervention Jane's BSP rate per 5 minutes was 1.14 over seven observations with a range of 0 to 3. Student time on-task averaged 76% with a range of 49% to 82%. Although Jane increased rates of BSP above 50% over baseline data, she was moved to Tier 2 because her BSP rates remained at 1 over five consecutive data points.

Tier 2. Jane increased the frequency of BSP to an average of 2.13 during Tier 2 intervention with a range of 0 to 3. Student average time on-task during Tier 2 was 57% with a range of 32% to 92%. Although the data showed a higher level of BSP, Jane's BSP frequency stayed consistent, with no increase over eight observations; therefore she was moved to Tier 3 to encourage increased BSP. During treatment fidelity checks the researcher discovered that Jane was not consistently videotaping her lessons, but she did so after resolving equipment concerns.

Tier 3. The average frequency of BSP during Tier 3 was 5.2 per 5 minutes with a range of 3 to 9. Student time on-task averaged 62% with a range of 39% to 87.

Social Validity

The results of the social validity questionnaire are presented in the following paragraph as well as Table 3. The three participants agreed with all of the statements but one. In response to the statement "professional development is more effective when it addresses needs of the faculty as a whole," one disagreed, one somewhat agreed, and the other agreed. All participants strongly agreed that BSP is an effective, feasible intervention to increase desired student behavior and that collaborative coaching is an effective tool for teacher improvement. All three indicated they would use self-monitoring sometimes, but they varied in their response to how often they would ask for assistance from a collaborative coach. One responded frequently, one responded sometimes and the third responded once in a while.

Additional comments reiterated-acceptance and confirmed efficacy of increasing BSP to improve student behavior:

* The BSP that I did on my class this year made a huge difference in the attitudes of my students. It didn't solve every problem, but it really had a strong impact on the behavior of the whole class as well as the targeted student. The downside to this was the timing. This is something that could have been implemented in the fall and saved lots of wasted time just with management. I will definitely incorporate this along with a few other things at the beginning of the year next year. I think it is really easy to get stuck in the habit of acknowledging the negative behaviors and overlook the positive behaviors of the students. I know that I didn't realize this until I started focusing on the positive behaviors. ... Now that I am comfortable with implementing BSP in my class, it isn't difficult or frustrating at all. ... Overall I learned some great ways to change the behaviors of the class and make my classroom a more positive environment.

* To be honest it made me very nervous to have other professionals observing me, but I learned through the process the value of praising specific behavior. I learned that it takes practice to see the behavior and then to give praise for the behavior. I know that I have improved on "seeing" the desired behavior and giving praise, and I will continue to improve this teaching technique.

Discussion

The main focus of this study was to examine the effects of tiered intervention on teachers' acquisition of a specific skill: Specifically, we examined teacher response to individualized professional development in respect to increasing BSP. Secondly, student response to teacher praise and teacher perceptions of the research efforts were evaluated. This study expands the findings of the Myers et al. study, as well as the research on effective professional development using video self-monitoring and coaching.

Baseline data on BSP rates revealed that teachers gave little to no behavior-specific praise statements, especially directed towards students they identified as disruptive. These findings are consistent with research on teacher-student interactions with little positive feedback or praise for appropriate conduct (Brophy 1981; Beaman & Wehldall, 2000; Sutherland et al., 2000). The baseline condition was followed by Tier 1 intervention--the school-wide faculty training on BSP. After this training, teachers were challenged to increase their frequency of BSP by 50%. (Those with a rate of 0 were challenged to increase their rate by 100%). All teachers at the faculty meetings, including the participants, committed verbally to do so for the school year.

Results indicated that participants' BSP did not increase consistently following the faculty training. During the Tier 1 condition two participants made slight improvements but rapidly returned to baseline, showing only slight effects between the independent and dependent variable. These results agree with research demonstrating that a one-time delivery of information is largely inadequate to change teacher behavior (Billingsley, 2005; Garet, Porter et al., 2001; Garet, Wayne et al., 2010; Guskey & Yoon, 2009). Similarly, these results extend comparable findings that a one-session faculty training so often used in school districts does not yield significant change in teacher behavior (Elmore, 2002; Fixsen et al., 2005; Myers et al., 2011; Sprick et al., 2006).

In contrast, when visual feedback (via video self-monitoring) was added during Tier 2, the frequency of BSP increased for all participants, especially for Gail, who examined her teaching and increased her BSP enough to require no additional support. Similarly, Anna reported, "I had no idea I said [a specific word] over and over as I teach. I need to change that right away."

Jane mentioned that she didn't realize she was favoring one side of her classroom; thus she made an effort to turn toward the students on the other side. These results support the use of video-taping in helping teachers notice classroom interactions as they develop effective teaching skills both as pre-service and as in-service teachers (Hennessy & Deaney, 2009; Hitchcock, Dowrick & Prater, 2003; Sherin & van Es, 2005). In addition to self-monitoring, coaching was folded into the three-tiered, professional development program to provide individualized support based on teacher need and personal choice. This decision to use coaching is supported by the body of educational professional development research which suggests that adult learning is more effective when it is contextual, ongoing, and classroom specific (Ackerman, 2008; Knight, 2009; Shidler, 2009; Oncharwi & Keengwe, 2008; Sprick et al., 2006).

As Anna and Jane received visits from the coach, their frequency of BSP increased. Jane received regular personal visits; however Anna did not due to her absences during scheduled visits and the work schedule of the coach. On days of no personal visits the coach contacted Anna by email. The frequency of BSP dropped on days of email correspondence and increased on days of personal visits, indicating the need for follow-up and accountability measures for teachers who do not respond to lower levels of support (Capizzi, Wehby, & Sand-mel, 2010; Hennessy & Deaney, 2009).

The coaching dynamics of this study highlight the difference between voluntary and assigned collaboration (Onchwari & Keengwe, 2008; Sprick et al., 2006). Teacher resistance was minimal, yet underlying defensiveness was evident during initial meetings with the participants. They may have felt that the principal was questioning their abilities, and resulting feelings of inadequacy could have impaired their teachability and their learning.

The ultimate purpose of informing teacher change is to impact student learning. In this study, student on-task behavior was also recorded in an effort to evaluate the possible effects of increasing BSP. Data for student time on task indicate similar patterns when viewed simultaneously with BSP rates: When the teacher praise rate was highly variable, the student on-task behavior was highly variable. Likewise, when the teacher praise was consistent and demonstrated a trend to increase, the student on-task behavior was steady and at a high level. Similar patterns in teacher-student data points may indicate a correlation between increased BSP and increased student on-task behavior, which supports findings from Sutherland et al. (2000) indicating that increased teacher praise results in increased student task engagement.

The study measured social validity to ascertain teacher perceptions of a responsive tiered framework of professional development. Teachers concurred that an individualized approach to professional development is more effective than a general whole-group approach. Additionally, all participants intended to continue self-monitoring to inform their practice. Participants responded differently in asking for the assistance of a collaborative coach; this finding further validates the importance of considering individual preferences and needs in teacher training (Myers et al., 2011).

Limitations and Future Research

The purpose of this study was to examine the relationship between the frequency of behavior-specific praise and a tiered intervention approach to professional development. Student on-task behavior was recorded secondary to this primary construct. Although student behavior did appear to follow similar patterns of teacher behavior, any implied relationship should be viewed with caution. Future research should systematically examine the causal relationship between teacher behavior and student behavior.

The participants were selected from a pool of teachers identified by their principal as needing assistance with difficult students. The participant attitudes were sometimes hesitant or even resistant, possibly affected by this selection process; however, they were cooperative, especially after the researcher showed interest in classroom activities and gave sincere, positive feedback on their interactions with students and their good teaching practices. Further research should consider implementation with teachers who may be more resistant to improving their classroom management skills.

Motivation to participate in interventions is an important part of coaching literature (Sprick et al., 2006). Nomination by their principals may have caused external rather than internal motivation for these teachers to participate. Sprick and colleagues maintain that if coaching or collaborative consultation between practitioners is to be optimally effective, it needs to be voluntary. Future research should broaden the scope by inviting all teachers in a school, to participate in a tiered professional development approach.

The researcher and observers were not part of the school faculty, which may have positively or negatively impacted teacher behavior. Guskey and Yoon (2009) maintained that outside experts can positively affect teacher improvement only as time is allotted for follow-up, demonstration and problem-solving activities. As ongoing professional development from outside sources is not financially feasible, studies should consider implementing this type of professional development using the existing training structures of the school or school district (e.g., district specialists, mentor teachers, school psychologists, and school principals).

In single-subject research studies, treatment fidelity is crucial in establishing functional relationships between the dependent and independent variables (Homer, Carr, & Halle, 2005). Inconsistencies in treatment fidelity were encountered during this study as well as in the Myers et al. (2011) study on which it was based. Frequent monitoring is key (Ardoin, 2006; Barnett et al., 2004). As the sole monitor of treatment fidelity, the researcher in this study discovered that two of the three participants were not consistently following listed procedures for Tier 2 and Tier 3 interventions. Simply asking the participants to sign the coaching log during each visit or to sign the treatment fidelity checklist to verify observance of the steps may increase fidelity of treatment. Future research should plan for an objective treatment fidelity measure, including inter-observer reliability, for use during the performance feedback intervention (Tier 2 in this study).

As with most single-subject research (Horner et al., 2005; Tawney & Gast, 1984), this study was necessarily conducted on a small scale limited to elementary-level teachers. A limited sample size, along with a participant pool including only white, female, middle-aged teachers affected the generalizability of the results. Replication of this study across grade levels and participant characteristics (e,g., gender, ethnicity, years of experience) may increase the external validity of the findings (Myers et al., 2011).

Another limitation of this study was lack of a maintenance phase. While the frequency of BSP did show an increasing trend and high stability for one participant, a maintenance phase of the study was not possible because the school year was almost over. Data from two of the three participants revealed that follow-up visits increased BSP, and without visits or contact to monitor teacher behavior, BSP returned to lower frequencies. Although Myers and colleagues (2011) included a maintenance phase during their similar study, they also found that without follow-up or monitoring, the frequency of BSP decreased. Future research should include a fade and maintenance phase to ensure skill acquisition (Myers et al., 2011).

Implications for Practice

Guskey and Yoon (2009) affirmed the importance of translating professional development into improved student outcomes and the necessity for its thoughtful planning. Methods taught should be individually responsive and continuous. Using a tiered continuum of ongoing teacher support accompanied by increased feedback has the potential to be individually responsive in its support of teacher skill acquisition, with critical follow-up embedded within the model structure (Guskey & Yoon, 2009).

This study confirms results from similar studies indicating that video self-monitoring, the second tier of the model, provides an accurate permanent product with a data set that meaningfully informs instruction, especially when accompanied by consultation from a mentor (Capizzi et al., 2010; Myers et al., 2011; Sherin & van Es, 2005). Performance feedback from video analysis allows teachers to think critically about their instruction and its effects on student achievement, making it a viable strategy to improve instructional practice (Capizzi et al., 2010; Colvin et al., 2009; Hitchcock, Dowrick & Prater, 2003). Large-scale implementation may include access to equipment for every classroom in a school. Group analysis of the recorded teaching segments, similar to a study by Sherin and van Es (2005), could be considered as an activity for professional learning communities.

One final implication from this study is that although research findings indicate that increasing praise results in increasing students' time on task and decreasing their disruptive behavior (Cherne, 2009; Sutherland et al., 2000), consensus on a prescribed number of praises per minute has not been reached by researchers and practitioners. Both Sutherland et al. (2000) and Myers et al. (2011) used six praise statements per 15-minute teaching segment as a standard for effective practice (Sutherland et al., 2000). This study considered the individual performance of the teacher and examined whether an incremental increase (50%<) from pre-intervention BPS rates and subsequent averages of each tier would affect student behavior. Preliminary outcomes demonstrated a relationship between teacher praise and student behavior; however, a specific prescribed praise rate cannot be specifically designated from the outcomes of this study. One implication is that praise is a low-cost, efficient, and effective strategy for teachers to promote positive student behavior. It requires minimal professional development effort and resources. Researchers should further examine the difference between using a predetermined number of praises per minute and requesting a percentage increase in determining at what point the student behavior is affected.

Conclusion

This study demonstrated that three elementary general educators increased their frequency of behavior-specific praise when provided with a continuum of performance feedback support that increased in intensity based on need. Thus a functional relationship was established between the independent and dependent variables. The increasing need for effective teachers intensifies the need for effective professional development systems. Methods of teacher training that provide a continuum of ongoing support, embedded evaluation, and follow-up are most effective. Researchers should continue to examine the effectiveness of providing a continuum of interventions to improve teacher skills in order to achieve the ultimate goal of improving student outcomes in academic, social, and behavioral areas.

References

Ackerman, D. J. (2008). Coaching as part of a pilot quality rating scale initiative: Challenges to--and supports for--the change-making process. Early Childhood Research & Practice, 10(2). Retrieved from http://ecrp.uiuc.edu/.

Ardoin, S. P. (2006). The response in response to intervention: Evaluating the utility of assessing maintenance of intervention effects. Psychology in the Schools, 43(6), 713-725.

Barnett, D. W., Daly, E. J., III, Jones, K. M., & Lentz, F. E., Jr. (2004). Response to intervention: Empirically based special service decisions from single-case designs of increasing and decreasing intensity. Journal of Special Education, 38(2), 66-79.

Beaman, R., & Wheldall, K. (2000). Teachers' use of approval and disapproval in the classroom. Educational Psychology, 20(4), 431-446.

Billingsley, B. S. (2005). Cultivating and keeping committed special education teachers: What principals and district leaders can do. Thousand Oaks, CA: Corwin Press.

Brophy, J. E. (1981). Teacher praise: A functional analysis. Review of Educational Research, 51, 5-32.

Burnett, P. C. (2002). Teacher praise and feedback and students' perceptions of the classroom environment. Educational Psychology, 22,5-15.

Capizzi, A. M., Wehby, J. H., & Sandmel, K. N. (2010). Enhancing mentoring of teacher candidates through consultative feedback and self-evaluation of instructional delivery. Teacher Education and Special Education, 33(3), 191-212.

Cheme, J. (2009). Effects of praise on student behavior in the classroom (Doctoral dissertation). Retrieved October 17,2010, from Dissertations & Theses: Full Text. (Publication No. AAT 3328300)

Colvin, G., Flannery, K.B., Sugai, G., & Monegan, J. (2009). Using observational data to provide performance feedback to teachers: A high school case study. Preventing School Failure, 53,95-104.

Coyne, M. D., Kame'enui, E. J., & Carnine, D. W. (2007). Effective teaching strategies that accommodate diverse learners. Upper Saddle River, NJ: Pearson Education.

Elmore, R. (2002). Bridging the gap between standards and achievement. Washington, DC: The Albert Shanker Institute.

Ferguson, E., & Houghton, S. (1992). The effects of contingent teacher praise, as specified by Canter's assertive discipline programme, on children's on-task behaviour. Educational Studies, 18, 83-93.

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis De la Parte Florida Mental Health Institute, The National Implementation Research Network. (FMHI Publication #231)

Fuchs, D., & Fuchs, L. S. (2006). Introduction to response to intervention: What, why, and how valid is it? Reading Research Quarterly, 41, 93-99.

Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915-945.

Garet, M., Wayne, A., Stancavage, F., Taylor, J., Walters, K., Song, M., ... & Doolittle, F. (2010). Middle school mathematics professional development impact study: Findings after the first year of implementation (NCEE 2010-4009). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Gresham, F. M. (2005). Response to intervention: An alternative means of identifying students as emotionally disturbed. Education and Treatment of Children, 28, 328-344.

Guskey, T. R., & Yoon, K. S. (2009). What works in professional development? Phi Delta Kappan, 90(7), 495-500.

Hennessy, S., & Deaney, R. (2009). The impact of collaborative video analysis by practitioners and researchers upon pedagogical thinking and practice: A follow-up study. Teachers and Teaching: Theory and Practice, 15(5), 617-638. doi: 10.1080/1350600903139621

Hitchcock, C. H., Dowrick, P. W., & Prater, M. A. (2003). Video selfmodeling intervention in school-based setting: A review. Remedial and Special Education, 24(1), 36-45.

Horner, R. H., Carr, E. G., & Halle, J. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165-179.

Joyce, B., & Showers, B. (1995). Student achievement through staff development: Fundamentals of school renewal. New York, NY: Longman.

Kalis, T. M., Vannest, K. J., & Parker, R. (2007). Praise counts: Using self-monitoring to increase effective teaching practices. Preventing School Failure, 51(3), 20-27.

Knight, J. (2009). Coaching: The key to translating research into practice lies in continuous, job-embedded learning with ongoing support. Journal of Staff Development, 30(1), 18-20.

MotivAider [Apparatus]. (2010). Thief River Falls, MN: Behavioral Dynamics, Inc. Information available at http://habitchange.com/motivaider.php

Myers, D. M., Simonsen, B., & Sugai, G. (2011). Increasing teachers' use of praise with a response-to-intervention approach. Education and Treatment of Children, 34(1), 36-59.

No Child Left Behind Act of 2001,20 U.S.C. [section] 6319 (2008).

Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., & Resetar, J. L. (2005). Treatment implementation following behavior consultation in schools: A comparison study of three follow-up strategies. School Psychology Review, 34(1), 87-106.

Onchwari, G., & Keengwe, J. (2008). The impact of a mentor-coaching model on teacher professional development. Early Childhood Education Journal, 36,19-24. doi:101007/s10643-007-0233-0

Peterson, D. S., Taylor, B. M., Burnham, B., & Schock, R. (2009). Reflective Coaching Conversations: A Missing Piece. The Reading Teacher 62(6). 500-509.

Sherin, M. G., & van Es, E. (2005). Using video to support teachers' ability to notice classroom interactions. Journal of Technology and Teacher Education, 13(3), 475-491.

Shidler, L. (2009). The impact of time spent coaching for teacher efficacy on student achievement. Early Childhood Education Journal, 36(5), 453-460.

Simonsen, B., Fairbanks, S., Briesch, A., Myers, D., & Sugai, G. (2008). Evidence-based practices in classroom management: Considerations for research to practice. Education and Treatment of Children, 31(3), 351-380.

Sprick, R., Knight, J., Reinke, W., & McKale, T. (2006). Coaching classroom management: Strategies and tools for administrators and coaches. Eugene, OR: Pacific Northwest Publishing.

Stichter, J. P., Lewis, T. J., Richter, M., Johnson, N. W., & Bradley, L. (2006). Assessing antecedent variables: The effects of instructional variables on student outcomes through in-service and peer coaching professional development models. Education and Treatment of Children, 29(4), 665-692.

Sugai, G. (2007). Promoting behavioral competence in schools: A commentary on exemplary practices. Psychology in the Schools, 44(1), 113-118. doi: 10.1002/pits.20210 Sutherland, K. S., Wehby, J. H., & Copeland, S. R. (2000). Effect of

varying rates of behavior-specific praise on the on-task behavior of students with emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders, 8, 2-8.

Tawney, J. W., & Gast, D. L., (1984). Single subject research in special education. Columbus, OH: Charles E. Merrill Publishing Company. doi:10.1177/109830079900100201

Yoon, K. S., Duncan, T., Lee, S. W., Scarloss, B., & Shapley, K. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007--No. 033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Centerfor Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Retrieved from http://ies.ed.gov/ncee/edlab.

Michele Terry Thompson

Michelle Marchant

Darlene Anderson

Mary Anne Prater

Gordon Gibb

Brigham Young University

Correspondence to Michelle Marchant, Department of Counseling Psychology and Special Education, 340-B MCKB, Brigham Young University, Provo, UT 84062; e-mail: michelle_marchant@byu.edu.
Table 1 Comparison of Myers et al. (2011) and Present Study

              Myers et al.      Thompson (2011)
              (2011)

Participant   Self-nominated    Principal-nominated
selection

Participant   SWPBS training    BSP rates < 50% of
criterion                       baseline

              P:R = reprimands
              greater than
              praise

Setting       Middle school in  Elementary schools,
              Northeast         Western US,

              US, implementing  no SWPBS
              SWPBS

Dependent     BSP, general      BSP, targeted STOT
variables     praise, P:R,
              composite STOT


Independent   RtI approach,     RtI approach,
variables     adjusting         adjusting level
              level of support  of support according
              according to      to teacher
              teacher           performance
              performance

Tier 1        SWPBS training    Faculty training
intervention  mastery           meeting on BSP

Tier 2        Weekly 10-min     Video self-monitoring
intervention  consultation      of BSP

Tier 3        Increased         Coaching
intervention  consultation      (consultation)

Movement      6 BSP per 15      BSP rates 50% >
criterion     min, P:R = 4:1    baseline

Note. SWPBS = schoolwide positive behavioral support
intervention plan; P:R = ratio of praise to reprimand;
BSP = behavior-specific praise; STOT = student time
ontask. Information for comparison is from "Increasing
Teachers' Use of Praise with a Response-to-Intervention
Approach," by D. M. Myers, B. Simonsen, and G. Sugai,
2011, Education and Treatment of Children, 34(1), pp.
36-59.


Table 2 Teacher Characteristics

Participant/School   Grade  Years teaching   Highest degree
                                             earned

Anna-A               4                   11  Bachelors
                                             in education

Gail-C               4                   13  Bachelors
                                             in education


Jane-B               3                    1  BS, working
                                             on ARL

Note. BS = bachelor of science; ARL = alternate route
to licensure.


Table 3 Social Validity Questionnaire Results

Question           Strongly  Disagree   Somewhat    Somewhat   Agree
                   disagree             disagree     agree

Faculty training                                            1       2
adequate to
increase BSP

Student changed                                                     2
behavior as
result of
increased BSP

BSP is an
effective
intervention

Increasing BSP is                                                   1
feasible and will
implement

Video                                                               2
sell-monitoring
is an effective
tool for
improving
classroom
management

Using a
collaborative
coach is an
effective tool
for teacher
improvement

Professional                                                        2
development is
more effective
when it addresses
individual needs
of each teacher

Professional                         1          1                   1
development is
more effective
when it addresses
needs of faculty
as whole

                     Almost  Once in a  Sometimes  Frequently  Almost
                      never      while                         always

How often will                                  3
you use video
self-monitoring

How often will                       1          1           1
you ask for a
collaborative
coach to improve
your teaching?

Question          Strongly agree

Faculty training
adequate to
increase BSP

Student changed                 1
behavior as
result of
increased BSP

BSP is an                       3
effective
intervention

Increasing BSP is               2
feasible and will
implement

Video                           1
sell-monitoring
is an effective
tool for
improving
classroom
management

Using a                         3
collaborative
coach is an
effective tool
for teacher
improvement

Professional                    1
development is
more effective
when it addresses
individual needs
of each teacher

Professional
development is
more effective
when it addresses
needs of faculty
as whole

                              Not
                       applicable

How often will
you use video
self-monitoring

How often will
you ask for a
collaborative
coach to improve
your teaching?
Gale Copyright:
Copyright 2012 Gale, Cengage Learning. All rights reserved.