Have we “Hattie” enough debate? Show me the data!

If one is careful not to take oneself too seriously, one can gain perspective in a 40-year career. Here is mine so far.

I am very tired of ideological arguments. I get that religion is ideological. I do not get that politics are.

It is impossible that every idea coming from a Democrat is wrong. It is impossible that every idea coming from a Republican is wrong.

I used to write a monthly column for the Deseret News in Salt Lake City. My first was a piece on the need for democracy to have the chaos of a good discussion. We have lost that ability.

In the Congress, it used to be true that Democrats and Republicans would scream at each other and then go have dinner together. No more.

It used to be that political issues could be discussed among friends. No more, unless everyone first shows her or his card proving their pre-agreement about every issue.

My wife and I have lost good friends over the politics of the last decade. Tragic.

One would think that education, the one chance we have with our treasured children to guide them to caring and competent citizenship, would be immune from ideological disagreements.

Of course, it is not. Three examples:

Whole language versus direct instruction is the first example. It is not possible that we as a field have not discovered the best ways to teach virtually all children to read proficiently. If one looks at the data, in fact, we have. But, because, some of the explicit approaches that work violate the ideology of many professors and some educators, we refuse to implement widely what works. Show me the data!

The use of the IQ/achievement discrepancy to identify students eligible for special education in the learning disabilities category is the second example. The IQ/achievement discrepancy sure looked scientific. When I was Utah’s state director of special ed (and I am a school psychologist), we spent hundreds of thousands of dollars on a discrepancy disk using a mile long regression formula from Cecil Reynolds.

In the early part of this century, Jack Fletcher, Matt Burns, and many others showed us unequivocally that the discrepancy model is not scientifically valid.

Did everyone immediately stop using it? Of course not. After all, RtI is not fully developed as an alternative. Besides that, we psychs have to have our WISC kits!

The RtI revolution is way past its tipping point and has been transformed into building a sustainable and coherent multi-tier system of supports (MTSS) for all students. More on MTSS later. Show me the data!

Site-based decision making is the third example. For years, we have tried to make site-based decision making work. Our ideological reliance on this idea has not raised achievement.

Marzano and Waters (2009) did a meta-analysis that found that site-based decision making does not work to raise achievement. What does work is for a school district to establish a limited number of non-negotiables and then to have schools innovate in the context of those non-negotiables.

Have we all listened? Oh no. If our ideology is site-based decision making, we keep doing it until we retire. This sort of adult-centered decision making is the biggest problem we have in education. Show me the data!

As a past master of corny titles, I think this one wins. “Have we ‘Hattie’ enough debate?” The reason for this cornball title is because of what I think is the most important contribution to education in my career.

John Hattie has recently (2009, 2012) published his epic scientific findings. Called Visible Learning, these books describe the results of the following methodology: 800 meta-analyses of 50,000 research articles including 150,000 effect sizes from a cumulative sample of 240 million students. Show me the data indeed!

Remember that an effect size of 0.2 or more is a significant effect.

Here are the top 10 influences on achievement from Hattie’s research:

Rank	Most Influence	Effect Size
1	Self-reported grades/Student expectations	1.44
2	Piagetian programs	1.28
3	Response to Intervention	1.07
4	Teacher credibility	0.90
5	Providing formative evaluation	0.90
6	Micro-teaching	0.88
7	Classroom discussion	0.82
8	Comprehensive interventions for learning disabled students	0.77
9	Teacher clarity	0.75
10	Teacher clarity	0.75

Please note that all of these effect sizes are well above 0.2. Please note that the third most influential factor is RtI. The fifth is providing formative evaluation. The eighth is comprehensive interventions for LD students.

To be complete, here are the bottom 15 influences on achievement:

Rank	Most Influence	Effect Size
136	Teacher subject matter knowledge	0.09
137	Changing school calendars/timetables	0.09
138	Out-of-school curricular experiences	0.09
139	Perceptual-motor programs	0.08
140	Whole language	0.06
141	Ethnic diversity of students	0.05
142	College halls of residence	0.05
143	Multi-grade/multi-age classes	0.04
144	Student control over learning	0.04
145	Open vs. traditional	0.01
146	Summer vacation	-0.02
147	Welfare policies	-0.12
148	Retention	-0.13
149	Television	-0.18
150	Mobility	-0.34

 

So, according to Hattie, whole language has an effect size of 0.06, student control over learning, 0.04, retention, –0.13.

 

Our ideologies should not predetermine our decisions, my friends and colleagues. If it works, keep doing it. If it doesn’t, we must stop.

 

This directive is the foundation for the MTSS initiative. Each of our students needs whatever she or he needs to be successful.

 

The only way to be sure that we provide “whatever” is to collect formative data about the effect of all of our instructional strategies on each of our students. Those that work, we continue. Those that don’t, we stop using.

 

Organizing these strategies into an MTSS (one districtwide system, many data based supports, directed by districtwide non-negotiables) is a proven transformational strategy that works. Take a look at the 2010 and 2011 Fullan books in the Kukic Collection included below for examples from all over the country.

 

Because of Hattie’s work, we know what works and what doesn’t. If we really treasure our students, can we do anything other than use these data to direct our efforts? I think not.

Steve Kukic's Collection (April, 2013)

 

Books:

Allain, J. (2008). The logistics of literacy intervention. Longmont, CO: Sopris West.

Archer, A.L. & Hughes, C.A. (2011). Explicit instruction. New York: Guilford Press.

Balfanz, R., Fox, J.H., Bridgeland, J.M., & McNaught, M. (2008). Grad Nation.

Bennis, W. & Biederman, P.W. (1997) Organizing genius. Reading, MA: Addison-Wesley.

Bossidy, L. & Charan, R. (2002). Execution. New York: Crown Business.

Brendtro, L.K., Brokenleg, M., & Van Bockern, S. (1990). Reclaiming youth at risk. Bloomington, IN: NES.

Bridges, W. (1991). Managing transitions. Reading, MA: Perseus Books.

Brown, J. (2006). The imperfect board member. San Francisco: Jossey-Bass.

Burns, M.K. & Gibbons, K.A. (2008). Implementing response-to-intervention in elementary and secondary schools. NYC: Routledge.

Byrne, Rhonda. (2006). The secret. New York: Atria Books.

Carnegie Council on Advancing Adolescent Literacy. (2010). Time to act: An agenda for advancing adolescent literacy. NYC: Carnegie Corporation of New York.

Christenson, S.L. & Sheridan, S.M. (2001). Schools and families. NYC: Guilford Press.

Clark, K.K. (1984). Grow deep, not just tall. St. Paul, MN: CEP. 952-454-1163

Collins, J. (2001). Good to great. New York: HarperCollins Publishers.

Collins, J. (2005). Good to great and the social sectors. Boulder, CO:

Collins, J.C. & Porras, J.I. (1997). Built to last. NY: HarperBusiness.

Cortiella, C. & Burnette, J. (2008). Challenging change. NY:

Covey, S.M.R. (2006). The speed of trust. NY: Simon & Schuster.

Covey, S.R. (2011). The 3^rd alternative. NY: Free Press.

Covey, S.R. (2004). The 8^th habit. NY: Free Press.

Covey, S.R. (1991). Principle Centered Leadership. NY: Fireside.

Covey, S.R. (1989). The seven habits of highly effective people. NY: Simon & Schuster.

Covey, S.R., Merrill, A.R., & Jones, D. (1998). The nature of leadership. SLC: Franklin Covey.

DuFour, R., DuFour, R., Eaker, R., & Karhanek, G. (2004). Whatever it takes. Bloomington, IN: Solution Tree.

Dweck, C. (2007). Mindset. New York: Ballantine Books.

Espin, C.A., McMaster, K.L., Rose, S., Wayman, M.M. (2012). A measure of success. Minneapolis: U of Minnesota Press.

Fixsen, D.L., Naoom, S.F., Balse, K.A., Friedman, R.M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa: University of South Florida.

Frankl, V.E. (1984). Man’s search for meaning. NY: Touchstone.

Fullan, M. (2013). Stratosphere. Toronto: Pearson Canada Inc.

Fullan, M. (2011). The moral imperative realized. Thousand Oaks, CA: Corwin.

Fullan, M. (2010). All systems go. Thousand Oaks, CA: Corwin.

Fullan, M. (2010). Motion Leadership. Thousand Oaks, CA: Corwin.

Fullan, M. (2008). The six secrets of change. New York: Jossey-Bass.

Fullan, M. (2003). Change forces with a vengeance. New York: RoutledgeFalmer.

Fullan, M. (2001). Leading in a culture of change. San Francisco: Jossey-Bass.

Fullan, M. (1999). Change Forces: The sequel. Philadelphia: The Falmer Press.

Fullan, M. (1993). Change forces. Philadelphia: The Falmer Press.

Fullan, M., Hill, P., & Crevola, C. (2006). Breakthrough. Thousand Oaks, CA: Corwin Press.

Gawande, A. (2009). The checklist manifesto. New York: Metropolitan Books.

Gay, G. (2000). Culturally responsive teaching. New York: Teachers College Press.

Gersten, R., Baker, S.K., Shanahan, T., Linan-Thompson, S., Collins, P., & Scarcella, R. (2007). Effective literacy and English language instruction for English learners in the elementary grades. www.centeroninstruction.org.

Gladwell, M. (2000). The tipping point. New York: Little, Brown, and Co.

Gladwell, M. (2008). Outliers. New York: Little, Brown, and Co.

Grant, J. & Forsten, C. (1999). If you’re riding a horse and it dies, get off. Petersborough, NH: Crystal Springs Books.

Griffiths, A., Parson, L.B., Burns, M.K., VanDerHeyden, A., & Tilly, W.D. (2007). Response to Intervention: Research for practice. Alexandria, VA: www.nasdse.org.

Hart, B. & Risley, T. (1995). Meaningful differences in the everyday experience of young American children. Baltimore: Paul H. Brookes.

Hattie, J. (2012). Visible learning for teachers. New York: Routledge.

Hattie, J.A.C. (2009). Visible learning. London: Routledge.

Havel, V. (1998). The art of the impossible. New York: Fromm International.

Heath, C. & Heath, D. (2007). Made to stick. New York: Random House.

Howard, P.K. (1996). The death of common sense. New York: Warner Books.

Jackson, P. (1995). Sacred hoops. New York: Hyperian.

James, J. (1996). Thinking in the future tense. New York: Simon & Schuster.

Johnson, R.A. (1991). Owning your own shadow. New York: Harper Collins.

Johnson, L.J., Zorn, D., Tam, K.Y., LaMontagne, M. & Johnson, S.A. (2003). Stakeholders’ views of factors that impact successful interagency collaboration. Exceptional Children, Vol. 69, No. 2, pp. 195-209.

Johnson, P.E. & Chrispeels, J.H. (2010). Linking the central office and its schools for reform. Educational Administration Quarterly. Vol. 46, No. 5, 738-775.

Keith, K.M. & Johnson, S. (2002). Anyway. New York: G.P. Putnam’s Sons.

Kouzes, J.M. & Posner, B.Z. (2003). Credibility. SF: Jossey-Bass.

Learning First Alliance. (2003). Beyond Islands of Excellence. www.learningfirst.org.

Lencioni, P. (2002). The five dysfunctions of a team. San Francisco: Jossey-Bass.

Lundin, S.C., Christensen, J., & Paul, H. (2000). Fish! New York: Hyperion.

MacKenzie, G. (1996). Orbiting the giant hairball. New York: Viking.

Marzano, R.J. (2007). The Art and Science of teaching. Alexandria: ASCD.

Marzano, R.J. & Waters, T. (2009). District leadership that works. Bloomington, IN: Solution Tree Press.

Marzano, R.J., Waters, T., & McNulty, B.A. (2005). School leadership that works. Alexandria, VA: ASCD.

McKinsey & Co. (2008). The economic impact of the achievement gap in America’s schools.

www.mckinsey.com/App_Media/Images/Page_Images/Offices/SocialSector/PDF/achievement_gap_report.pdf

Millman, D. (1984). The way of the peaceful warrior. Tiburon, CA: H.J. Kramer Inc.

Murphy, M. (1997-originally 1973). Golf in the kingdom. New York: Penguin.

National Association State Directors of Special Education. (2008). Blueprints for Implementation of Response to Intervention. Alexandria, VA: NASDSE.

NASDSE. (2005). Response to Intervention. Alexandria, VA: NASDSE.

National Council for the Education of Black Children (NCEBC). Blueprint IV. www.ncebc.org.

Olson, R. (2009). Don’t be such a scientist. Washington, DC: Island Press.

Pagano, B. & Pagano, E. (2004). The transparency edge. New York: McGraw-Hill.

Palmer, P. (2000). Let your life speak. San Francisco: Jossey-Bass.

Palmer, P. (1998). The courage to teach. San Francisco: Jossey-Bass.

Palmer, P. (1993). To know as we are known. San Francisco: Harper.

Patterson, K., Grenny, J.,Maxfield, D., McMillan, R., Switzer, A. (2008). Influencer. New York: McGraw-Hill.

Pausch, R. (2008). The last lecture. NYC: Hyperion.

Payne, C.M. (2008). So much reform, so little change. Cambridge, MA: Harvard Education Press.

Peck, M.S. (1999). Golf and the spirit. New York: Harmony Books.

Peters, T. & Waterman, R. (2004, 1982). In search of excellence. New York: HarperCollins.

Peters, T. (1987). Thriving on chaos. NY: Alfred A. Knopf.

Pirsig, R.M. (1974). Zen and the art of motorcycle maintenance.  New York: William Morrow.

Rath, T. & Clifton, D.O. (2005). How full is my bucket? NY: Gallup Press.

Reeves, D.B. (2006). The learning leader. Alexandria: ASCD.

Ruiz, D.M. (1997). The four agreements. San Rafael, CA: Amber-Allen Publishing.

Scammacca, N., Roberts, G., Vaughn, S., Edmonds, M., Wexler, J., Reutebuch, C.K., & Torgesen, J.K. (2007),

Interventions for adolescent readers. Portsmouth, NH: RMC Research Corporation, Center on Instruction. www.centeroninstruction.org

Schmoker, M. (2006). Results now. Alexandria, VA: ASCD.

Secretan, L.H.K. (1997). Reclaiming higher ground. Toronto: MacMillan Canada.

Sergiovanni, T.J. (1994). Building community in schools. San Francisco: Jossey-Bass.

Shaywitz, S. (2003). Overcoming dyslexia. New York: Knopf.

Shinn, M.R. & Walker, H.M. (2010). Interventions. Bethesda, MD: NASP Publications.

Singleton, G.E. & Linton, C. (2006). Courageous conversations about race. Thousand Oaks, CA: Corwin Press.

Tschannen-Moran, M. (2004). Trust matters: leadership for successful schools. San Francisco: Jossey-Bass.

Wheatley, M.J. & Kellner-Rogers, M. (1996). A simpler way. San Francisco: Berrett-Koehler.

Wooden, J. (1997). Wooden. New York: Contemporary Books.

Wooden, J. (1988). They call me coach. Chicago, IL: Contemporary Books.

Watzlawick, P. (1983). The situation is hopeless, but not serious. New York: W.W. Norton.

X, Malcom. (1964). The autobiography of Malcom X. New York: Ballantine Books.

Zenger, J.H. & Folkman, J. (2002). The extraordinary leader. New York: McGraw-Hill.

Zogby, J. (2008). The way we’ll be. New York: Random House.

Websites:

1000 Awesome Things

Alliance for Excellent Education

Center on Instruction

Center on Teaching and Learning: Big Ideas on Reading

Children’s Defense Fund

Collaborative for Academic, Social, and Emotional Learning (CASEL)

Colorado Department of Education

Common Core State Standards Initiative

Data Accountability Center (DAC)

EasyCBM

Explicit Instruction

Florida Center for Reading Research

Florida’s Multi-tier System of Supports

Florida’s Problem Solving & Response to Intervention Project

Heartland Area Education Agency

Illinois ASPIRE (Alliance for School-based Problem-solving and Intervention Resources in Education)

Intervention Central

Kansas Multi-tier System of Supports

Los Angeles Unified School District Response to Instruction and Intervention (RtI2)

Manhattan Institute for Policy Research

Meadows Center for Preventing Educational Risk

Michigan's Integrated Behavior and Learning Support Initiative (MiBLSi)

Motion Leadership: Michael Fullan

Moving Your Numbers

National Association of State Directors of Special Education

National Center for Learning Disabilities

National Center on Intensive Intervention

National Center on Response to Intervention

National Council on Educating Black Children

National Dropout Prevention Center for Students with Disabilities

National High School Center

National Longitudinal Transition Study-2

NCLD’s A Parent’s Guide to Response-to-Intervention

OSEP Technical Assistance Center on Positive Behavior Interventions and Supports

Research Institute on Progress Monitoring

Response to Intervention Guidance for New York State School Districts

RTI Action Network

Search Institute

TED: Ideas worth spreading

The Lexile Framework for Reading

U.S. Department of Education: Recovery Act

Wisconsin's Vision for Response to Intervention