Research Articles and Papers from ASSC




A Paper Presented at the Annual Meeting of the American Educational Research Association, San Diego, April 2004


Does Seeking to Create a Better Classroom Climate lead to Student Success or Improved Teaching? Examining the Relationship Between Pedagogical Choices and Classroom Climate in Urban Secondary Schools

There is growing evidence to suggest a strong link between achieving student affective outcomes and corresponding achievement outcomes (Hoy & Hannum, 1997; Hawkins, 1997; Levin & Wiens, 2003; Ryan & Patrick, 2001; Sweetland & Hoy, 2000; Wang, Haertel & Walberg, 1997). Moreover, research shows that students experience gains in both academic achievement and academic self-concept as teachers seek to develop students' academic agency (Jackson, 2003). Jackson describes student academic agency as instructional approaches that are "identity-sensitive." More than a decade of research supports the affective and academic benefits of what Glasser (1990) refered to as "needs-satisfying" curriculum. Building on Glasser's work, Patterson (2003) found that the use of problem-based curricula that challenge students to grow as reflective learners and in their collective functioning result in the development of students who are self-directed, critical thinking learners. He concludes that schools "must include a vision of what kind of learners we are trying to develop and how we can assist students to become what we envisioned. We must put the educational needs of the students above the comfort of the adults." Programs such as Facing History and Ourselves (FHAO) offer teachers a powerful history curriculum as well as problem-based pedagogical format that promotes both "identity-sensitivity" and what Schultz, Barr & Selman (2001) term "relational maturity" resulting in an increase in student agency and the group's collective functioning. Nevertheless, the effectiveness of such programs will ultimately vary substantially depending on the teacher, their values, the pedagogy used before and during any such unit, various contextual variables and the pre-existing classroom climate (Sweetland & Hoy, 2000; Wang, 1997).

Curricular approaches that promote what could be defined as critical pedagogy, and/or those that promote the collective social and emotional intelligence of students are typically endorsed on moral and altruistic grounds, yet rarely discussed as embodying what could be considered "effective teaching." Shindler, Taylor, Jones & Cadenas (2003) found that efforts toward what could be termed "intentional climates" that were defined by a transformational or liberating practice did appear to produce greater student achievement. This research suggests that the choice that often seems to be presented in teacher education (implicitly, if not explicitly) between either teaching for depth of understanding, meaning making and affective growth, or that of preparation for demonstration of competency, may be a false choice. In fact, attaining high-levels of academic competency may be more likely within a classroom climate that is intentional and "needs-satisfying."

Purpose and Guiding Questions for the Study
The study was a pilot effort to examine the relationships among a series of factors within the practice of a teacher who is attempting to implement the Facing History and Ourselves (FHAO) curriculum into his/her classroom. The factors included the content being taught, the pedagogical choices made, the "intentionality" of the efforts related to classroom climate development, and their effects on student achievement and classroom climate. The pilot study attempted to examine four primary questions:

1. Do the pedagogical choices made by the teacher affect the degree to which the goals of a curriculum such as FHAO can be met?
2. How do certain pedagogical choices affect the quality of classroom climate?
3. What is the relationship between the degree of "intentionality" in the development of classroom climate and student learning?
4. What long-term impact does incorporating a critical curriculum such as Facing History and Ourselves have on a teacher?

Study Methods
The pilot study employed a qualitative emergent design to follow 2 teachers into the field that had recently completed a Facing History and Ourselves (FHAO) Training Institute. Participants were examined while in the midst of units that lasted from 2-4 weeks. Teacher participants were purposively sampled to be representative of the most "successful FHAO teachers." Both teacher participants were experienced teachers of over 5 years and held graduate degrees. Two additional classes of the same course taught by non-FHAO teachers with equivalent population of students were selected from each of the schools to act as control samples.

Data collection included observation of teacher practice during the unit, interviews with the teacher participants, a survey of FHAO goal accomplishment of both teacher and student participants, a survey of classroom climate by students, and focus group interviews of students near the end of the unit. The survey used to assess classroom climate incorporated an analytic trait format. The instrument format is depicted in below.

Example A: Analytic Trait Scale Instrument Item Example:
High	Middle	Low
Teacher-student interactions could be typically described as supportive and respectful.	Teacher-student interactions could be typically described as fair but teacher-dominated.	Teacher-student interactions are mostly teacher-dominated and reactive.

In a study of the efficacy of the ASSC system in an urban setting (Shindler, Taylor, Jones & Cadenas, 2003), significant advantages for a participant-driven, analytic-scale system were observed. The analytic-scale (rubric) instrument demonstrated greater soundness (validity, reliability, efficiency and benefit) than traditional inventories.

Data analysis was conducted in 3 phases.

• Phase 1 - Study protocol development
  • Observational data was collected from the participants in School 1, an analytical framework was established for further data collection in phases 2 and 3.
  • Climate was confirmed as a salient factor and incorporated into subsequent phases.
  • A "Social Studies Outcome" survey was developed to assess student report of success related to the intended outcomes of the FHAO program.
  • A focus group protocol was developed for phase 2.
  
  
• Phase 2 - Data collection from Participant Teacher 1 including:
  • In-depth examination of each area of the climate inventory
  • In-depth examination of each item on the FHAO survey - with a special emphasis on judging causality for ratings
  • Explore a analytical framework for a "successful FHAO teacher." The framework includes 3 factors
    1. The relationship of the teacher to the students
    2. The pedagogy used before and during the unit
    3. The FHAO curriculum content
  
  
• Phase 3, the data collection protocol was repeated with the participant and control classes in school 2.

Results
The results of the survey data indicated that the Facing History and Ourselves (FHAO) teacher participants showed a significantly better performance across all measures on both classroom climate and FHAO/Social Studies outcomes. In school 1, the FHAO participant teacher's class rated the overall classroom climate as a 7.1 (falling in the 3 or high level) as compared to the control class rating of 5.4 (falling in the middle or 2 level). Likewise the students rated the FHAO teacher as much more successful in accomplishing goals such as providing a curriculum that "addressed issues of race and membership" (1.2 or "strongly agree" rating as compared to a 2.3 or "agree to neutral" rating), and providing a curriculum that encouraged students to take an active role as citizens (2.0 or "agree rating vs. a 3.2 or "neutral to disagree rating).

In School 2, the findings were comparable. The FHAO teacher's class rated the climate mean as 7.5 overall (e.g., "middle high level"/ 3 points) or a full level higher than the control class (X=4.5). Likewise on the FHAO/Social Studies Outcome ratings the FHAO teacher scored a 2.1 or a mean of an "Agree" response versus a 2.9 or a mean response of a "neutral' response for the control class.

This data suggests a clear relationship between the quality of the pedagogy and student perception of the success of the course. Students rated the curriculum as more effective in the FHAO classes. Yet, it is not clear if and/or to what extent the teaching methods contributed to the students' perceptions that the curriculum was so meaningful or if it was that the curriculum could be seen as enhancing the perception of the methods. Nevertheless, participants in each focus group suggested the there was an interrelationship. This relationship could be seen in the correlation of scales 2 and 4 on the climate inventory. For both FHAO teacher participants the perceived effectiveness of their teaching methods (X=7.0,7.6) was related to high scores in the culture of the class (X=7.1,7.5). This relationship could also be seen in the control groups. However in the control groups the ratings were significantly lower (see Table 1). A causal relationship between the quality of the instruction and the way they felt about the culture of the class and their relationships to one another was reported by students in the focus group interviews.

Students also reported a strong influence of the methods used on the efficacy and agency of the curriculum and content being covered. In both schools, the students suggested that the curriculum would have been much less meaningful without being delivered within a methodological framework that they experienced as successful. But the students also suggested that given the high level of trust created, the high quality of the relationship with the teacher and the engaging instructional climate, the pedagogical methods were enhanced by a curriculum that contributed to the relevance and interest level of their work and learning experience.

Table 1: Classroom Climate Survey Comparison of Means for Participant and Control samples

	S1 FHAO (N=33)	S1 control (N=28)	S2 FHAO (N=34)	S2 control (N=26)
Scale 1 Student Interactions	6.5	5.4	7.1	4.3
Scale 2 Discipline Environment	7.7	5.5	8.0	4.8
Scale 3 Learning/Assessment	7.0	5.6	7.6	4.7
Scale 4 Attitude & Culture	7.1	5.1	7.5	4.4
Sub-scale A Relationship	7.7	5.4	8.2	4.5
Sub-scale B Clarity	7.2	5.7	7.5	4.9
Sub-scale C Community	7.4	5.5	7.5	4.2
Sub-scale D Diversity	7.8	5.4	7.8	4.4
Overall Mean	7.1	5.4	7.5	4.5

Scores are derived from a mean rating for each item. Items are rated 1-3 for low quality rating,
4-6 for middle level ratings and 7-9 for high level ratings (see Example A above).

Table 2: Facing History and Ourselves/Social Studies Goals Survey Results

	S1 FHAO (N=33)	S1 control (N=28)	S2 FHAO (N=34)	S2 control (N=26)
1. Self Understanding	2.4	2.6	2.8	3.6
2. Addresses issues of race, class, membership	1.2	2.3	2.0	2.6
3. encourages citizen participation	2.0	3.2	1.9	3.0
4. understand broader implications of history	1.8	2.0	1.7	2.5
5. builds classroom community	2.4	2.6	2.0	3.1
6. feels like "authentic" history	1.5	2.2	2.0	2.4
7. effects my behavior outside of class	1.8	2.0	2.4	2.8
Overall Mean	1.9	2.4	2.1	2.9

Scores are derived from a mean rating for each of the 7 items. 1=Strongly Agree, 2=Agree,
3=Neutral, 4=Disagree, 5=Strongly Disagree.

The results of the focus group interview helped clarify the causality of the survey data. Students were enthusiastically positive about the methods of the 2 FHAO teachers. Neither of the teachers experienced any classroom management problems even though the students reported that classroom discipline was a big issue in most of their other classes. This discrepancy was confirmed by researcher observations as well. The Students in both FHAO classes experienced a high degree of "community" while reporting very little of it their other classes. The focus group interviews provided more support for the significance of the relationships that were formed with each of the FHAO teachers. Asked why there were no discipline problems, why all students seemed to be prepared, motivated and excited to be in class, unanimously the students reported that it was due to what was taking place in the class and especially to their relationship with the teacher. They sited examples of the teacher asking them about their lives outside of school, taking a sincere interest in them as individuals and treating them with respect.

Comparing the methods incorporated by the FHAO teachers to the control teachers, there were clear differences. The FHAO teachers used group work purposefully and taught the students how to function in collectives. The FHAO teachers found engaging interactive methods to present material such as presentations, role-plays, debates, and discussions. In both FHAO classes, the discussions that were observed were striking in that nearly all students felt welcome to participate, were engaged, and projected a supportive, egalitarian affect to their peers. This type of climate was not the case in other classes observed at either of the 2 schools, in fact it could be characterized as exceptional.

Within the context of this supportive interpersonal climate and the engaging pedagogical environment, the evocative and personally and socially challenging content of the FHAO curriculum flourished. The students took the material very seriously and were very respectful to the issues they were addressing. Absent was any hint of the students ridiculing or making light of those they were studying. It would have been expected to see some students approaching the material with a lack of maturity or compensating for a sense of "group/collective guilt" by joking about some of the emotionally challenging material. Yet, only a mature and respectful affect was evidenced.

Asked to describe what it was that made the class "successful," the students suggested that the teacher was the most significant variable, next was the methodology and then came the curriculum they selected to teach. Asked to give percentages for each, both FHAO classes reported about the same proportions. It averaged out to about 50,35,15 - relationship, pedagogy, curriculum. However, they felt that all 3 were interrelated, and critical to the level of success that there had experienced. In school 2, where the teacher used less formal FHAO curriculum, the students saw less of a critical role of the content for what they felt made the class successful. In school 1, where the students had been exposed to a longer unit and a more substantial and sustained use of the FHAO curriculum, the students seemed to feel the curriculum content was more responsible for what might be referred to as the "transformational quality" that they experienced in the course as a whole.

An indication of the effect the teacher's commitment in school 1 to the FHAO curriculum had could be seen in the rating of the class for item 2 in the Social Studies Survey, "The content of the class effectively addresses issues of race, membership and identity." This FHAO class overwhelmingly rated this item with a "strongly agree" response (X=1.2), compared to an average response of "agree" in the FHAO class in school 2 (X=2.0), and much more neutral responses in the control groups (X=2.3,2.6). So while both FHAO teachers were rated as highly successful on all accounts by their students, and their methods translated into environments where students approached the material with care, thought and inquiring minds, the data seems to suggest that more time spent engaged in the FHAO curriculum did appear to produce more success in terms of the intended FHAO outcomes.

When each FHAO class was asked how they would have hypothetically responded to such a challenging curriculum dealing with race, class and membership within a class where they did not have the kind of relationship with their teachers that they currently experienced, and if the teaching methods used were characterized by a traditional "sit and get," lecture and test format, the students suggested that it would make the curriculum essentially academic. They felt that it would be, in effect, disconnected and not nearly as meaningful to their lives as what they currently experienced.

Consistent with its effects on the students, the data suggests that the FHAO curriculum cannot be examined outside of its effects on the teachers and their practice. Both teachers reported growing pedagogically as a result of their involvement with FHAO. While each participant seemed committed to the creation of a highly effective class regardless of the curriculum they used, the FHAO curriculum provided a venue for each to examine their methods and their materials and make improvements. Each reported that they were more purposeful in how they tried to bring about affective domain learning outcomes with their students. They also tended to use more primary documents and felt more comfortable teaching for a greater depth of understanding rather than accepting such a heavy (district driven) emphasis on content coverage.

Discussion
The results of the study suggest that these 2 FHAO teachers successfully achieved "intentional climates" in their classes. The results also suggest that in the control group classrooms what was created could best be characterized as "accidental climates" (Shindler, Jones, Taylor & Cedenas, 2003). Within these intentional climates, a sense of community, academic rigor and relational maturity were better able to take root. It appears that the intentionality of the climates, the quality of the relationships and use of highly effective pedagogy were all critical to promoting a receptivity, depth of processing and a seriousness of treatment with the FHAO content. In this climate, the FHAO curriculum flourished. It is not clear what would happen if a teacher attempted to incorporate the challenging FHAO curriculum in an unintentional/accidental climate, where there were not the relational or pedagogical requisite conditions present. Because of the relational maturity (Schultz, Barr & Selman, 2001) required for FHAO to gain a receptivity, in the absence of these requisite conditions it is very possible that the FHAO curriculum may be trivialized by students who are not emotionally prepared as individuals or as a collective to treat the material seriously. While this assertion was suggested by some student participants, and appears to make sense, it would require further study to confirm.

This pilot study seems to supports the interdependency of factors related to the effects of pedagogy, the relationship environment, the intentionality of the climate, the quality of the curriculum and the level of student learning. Students in the FHAO classes, while similar to those in the control samples, reported learning more and having a higher degree of investment in their learning. These results support the contention that each of these areas needs to be approached within the broader whole. Moreover, efforts to improve one of these areas or to implement a program that does not consider the holistic relationships among the factors may be inherently limited in its efficacy.

The study may shed light on the need to rethink the starting point in the teacher reform process. As Wagner (2003) and Levin (2003) report, other countries have found a great deal of success beginning the school restructuring process by defining the values and dispositions that they want students to possess at the end of their educational experience, rather than focusing on such outcomes as standardized test gains. If an emphasis on developing social and emotional capital in students is the critical piece to student success, then we as a field may want to shift the way we assess both micro and macro educational efforts.

Since currently classroom climate is relegated to a peripheral concern in the process of teacher development, mostly taught in connection to classroom management and rarely as part of the pedagogical domain, this study may provide some support for a more prominent place for it in the teacher preparation process. If in fact there exists an interrelatedness of pedagogy and curriculum that creates "good climate" which in turn leads to better results with students, then understanding the process of constructing "good climate" early in one's teaching career is essential for high teaching performance.

Conclusion
The current curricular climate in schools often creates a mindset in new teachers that they must choose between student academic achievement and student social and emotional development. There is growing research to suggest that this choice is not inherently necessary or valid. But the essence of bringing these two domains together seems to be more a matter of the "hows" than the "whats" of teaching. This limited study supports the growing insight into how schools might approach the difficult task of attaining high standards, develop emotionally and socially healthy students, and supporing sound curricular choices, and the place an intentional approach to the area of classroom climate and the added dimensions provided by a curriculum such as Facing History and Ourselves can play in that effort.


References