Educators struggle with a number of questions regarding teaching methods, including:
- What educational policies have the greatest impact on students?
- What influences students to achieve?
- What are best practices for teachers yield the best results?
Roughly 78 billion is the estimated dollar amount invested in education by the United States according to market analysts (2014). So, understanding how well this enormous investment in education is working requires a new kind of calculation in order to answer these questions.
Developing that new kind of calculation is where Australian educator and researcher John Hattie has focused his research. In his inaugural lecture at the University of Auckland as far back as 1999, Hattie announced the three principles that would guide his research:
"We need to make relative statements about what impacts on student work;
We need estimates of magnitude as well as statistical significance - it is not good enough to say that this works because lots of people use it etc., but that this works because of the magnitude of impact;
We need to be building a model based on these relative magnitudes of effects."
The model he proposed in that lecture has grown to become a ranking system of influencers and their effects in education using meta-analyses, or groups of studies, in education. The meta-analyses he used came from all over the globe, and his method in developing the ranking system was first explained with the publication of his book Visible Learning in 2009. Hattie noted that the title of his book was selected to help teachers "become evaluators of their own teaching” with the objective of giving teachers a better understanding of the positive or negative effects on student learning:
"Visible Teaching and Learning occurs when teachers see learning through the eyes of students and help them become their own teachers."
Hattie used the data from multiple meta-analyses in order to get a "pooled estimate" or measure of an effect on student learning. For example, he used sets of meta-analyses on the effect of vocabulary programs on student learning as well as sets of meta-analyses on the effect of preterm birth weight on student learning.
Hattie's system of gathering data from multiple educational studies and reducing that data into pooled estimates allowed him to rate the different influences on student learning according to their effects in the same manner, whether they show negative effects or positive effects. For example, Hattie ranked studies that showed the effects of classroom discussions, problem-solving, and acceleration as well as studies that showed the impact of retention, television, and summer vacation on student learning. In order to categorize these effects by groups, Hattie organized the influences into six areas:
- The student
- The home
- The school
- The curricula
- The teacher
- Teaching and learning approaches
Aggregating the data that was generated from these meta-analyses, Hattie determined the size of the effect each influence had on student learning. The size effect could be numerically converted for purposes of comparison, for example, an influencer's effect size of 0 shows that the influence has no effect on student achievement. The greater the size of the effect, the greater the influence. In the 2009 edition of Visible Learning, Hattie suggested that an effect size of 0,2 could be relatively small, while an effect size of 0,6 could be large. It was the effect size of 0,4, a numerical conversion that Hattie termed as his “hinge point,” that became the effect size average. In the 2015 Visible Learning, Hattie rated influence effects by increasing the number of meta-analyses from 800 to 1200. He repeated the method of ranking influencers using the “hinge point” measurement which allowed him to rank the effects of 195 influences on a scale. The Visible Learning website has several interactive graphics to illustrate these influences.
The number one influencer at the top of the 2015 study is an effect labeled “teacher estimates of achievement." This category, new to the ranking list, has been given a ranking value of 1,62, calculated at four times the effect of the average influencer. This rating reflects the accuracy of an individual teacher's knowledge of students in his or her classes and how that knowledge determines the kinds of classroom activities and materials as well as the difficulty of the tasks assigned. A teacher's estimates of achievement can also influence the questioning strategies and the student groupings used in class as well as the teaching strategies selected.
It is, however, the number two influencer, collective teacher efficacy, that holds an even greater promise for improving student achievement. This influencer means harnessing the power of the group to bring out the full potential of students and educators in schools.
It should be noted that Hattie is not the first to point out the importance of collective teacher efficacy. He is the one who rated it as having an effect ranking of 1.57, almost four times the average influence. Back in 2000, educational researchers Goddard, Hoy, and Hoy advanced this idea, stating that “collective teacher efficacy shapes the normative environment of schools” and that the “perceptions of teachers in a school that the efforts of the faculty as a whole will have a positive effect on students.” In short, they found that “teachers in this school can get through to the most difficult students.”
Rather than rely on the individual teacher, collective teacher efficacy is a factor that can be manipulated at a whole school level. Researcher Michael Fullen and Andy Hargreaves in their article Leaning Forward: Bringing the Profession Back In note several factors that must be present including:
- Teacher autonomy to take on the specific leadership roles with opportunities to participate in making decisions on school-wide issues
- Teachers are allowed to collaboratively develop and communicate mutual goals that are clear and specific
- Teachers are committed to the goals
- Teachers work as a team transparently without judgment
- Teachers work as a team to collect specific evidence to determine growth
- Leadership acts responsively to all stakeholders and show concern and respect for their staff.
When these factors are present, one of the outcomes is that collective teacher efficacy helps all teachers understand their significant impact on student results. There is also the benefit of stopping teachers from using other factors (e.g. home life, socio-economic status, motivation) as an excuse for low achievement.
Way at the other end of the Hattie ranking spectrum, the bottom, the influencer of depression is given an effect score of -,42. Sharing space at the bottom of the Visible Learning Ladder are the influencers mobility (-,34) home corporal punishment (-,33), television (-,18), and retention (-,17). Summer vacation, a much-beloved institution, is also negatively ranked at -,02.
In concluding his inaugural address almost twenty years ago, Hattie pledged to use the best statistical modeling, as well as to conduct meta-analyses to achieve integration, perspective, and magnitude of effects. For teachers, he pledged to provide evidence that determined the differences between experienced and expert teachers as well as to assess the teaching methods that increase the probability of impact on student learning.
Two editions of Visible Learning are the product of the pledges Hattie made in determining what works in education. His research can help teachers see better how their students learn best. His work is also a guide for how to best invest in education; a review of 195 influencers that can be better targeted by statistical significance for billions in investment… 78 billion to start.