In science, we build a firm foundation and then we keep renovating the house. We find interesting results, we are fascinated by them, we don’t always trust them, so we go back and replicate them. We also challenge them by asking, where will this not work? When does the effect go away? How can we use better methods to test our theories?
As part of this process, scientists ask each other questions. Recently, other scientists asked us some questions about three of our papers. We took this very seriously, carefully considered each inquiry, delved into the studies again (in some cases reanalyzing the data), and prepared three documents, each detailing our process and our findings (here, here, and here). In each case, we showed that the conclusions reached in the paper were sound. But, as with anything that helps make science better, we were grateful for the questions because they pointed out areas for improvement or clarity, and because we believe in open science.
It is important however to consider these questions in light of a large body of work. The growth mindset story does not rest on a handful of isolated studies. Research in this area has been ongoing for 30 years and the field has amassed a large body of work. A meta-analysis published in 2013 found 113 studies conducted by many authors and concluded that mindsets are a significant factor in people’s self-regulation toward goals.
Meta-analyses are helpful, but not the final word. Government data collected at a country level—all the 10th grade students in the country of Chile (over 160,000)—showed that holding a growth mindset predicted academic achievement at every socioeconomic level. Recently, the state of California, collecting data from over 100,000 middle schoolers, found that students’ mindsets were a good predictor of their test scores. And this doesn’t include many experimental studies that have carefully oriented children (or adults) toward different mindsets and found effects on outcomes. It is highly unlikely that mindset is a phantom phenomenon.
Despite this evidence, it can still seem surprising when a mindset intervention changes students’ grades over time. We too have been surprised by this and treat our own findings with healthy skepticism. That is why we have been obsessed with doing larger and better studies to replicate and understand them.
Let me illustrate our scientific process.
We and others had published several mindset interventions, in which students are taught a growth mindset (the idea that intelligence can be developed) and then followed to see if their academic achievement is greater than students who were in a control condition. We have done this through specially designed in-person workshops, and more recently we have delivered them online directly to students in their schools. The results were very encouraging but we were not satisfied with them. For example, the effects were modest and were obtained only for the lower-achieving students, and the samples were samples of convenience, not random samples. We also didn’t know much about the circumstances under which the programs were most effective and least effective.
So, in 2014-2015, we spent a great deal of time developing a higher quality online program, including pilot studies with over 13,000 participants. In 2015-2016, in a study led by David Yeager, the online intervention was delivered to approximately 9,000 students making the transition to high school, with another 9,000 students in an active control group. Several things are noteworthy. The hypotheses and analyses for this study were pre-registered, meaning we said what counted as “success” or “failure” before we looked at the data, which is becoming best practice in science. Moreover, the study was conducted by an independent, disinterested research organization. The data will be independently analyzed by a network of sociologists and economists. Finally, the sample was a random sample of students across the U.S. entering high school (with certain groups oversampled). To our knowledge, this has not been done for any intervention in education in decades.
To help us understand when and for whom the intervention did and didn’t work, we also obtained a great deal of information about each school and about the math classes in each school. In 2016-2017, the main outcome data (academic grades) are being obtained from the schools. So far, the pre-registered analyses show that the online program changes students’ immediate behavior and mindsets, and does so consistently across schools and achievement levels. If this nation-wide study yields beneficial effects, we plan to release the online program to schools (and researchers) free of charge. We are also sharing our materials with independent replicators, and one group of economists has already replicated our findings in Norway.
This national study is a prime example of a rigorous scientific process that tries to understand when to expect and when not to expect mindset intervention effects. We will then go on to study how one might foster these effects more widely.
Importantly, although an online intervention is relatively easy to deliver, it was not easy to develop. It took us a long time to craft our most recent online intervention. We learned how to build in a lot of motivation for students. The ongoing intervention was very carefully crafted to ensure that students felt respected (they are helping us develop the program rather than being helped by us) and involved (we periodically ask them for their opinions and feedback). We also took steps to help them internalize the growth-mindset message (e.g., by having them write a letter to a struggling student, mentoring the student in terms of growth-mindset principles). Finally, we took great pains to ensure that the growth-mindset message was compelling by providing neuroscience findings, clear application of the principles to the students’ lives, testimonials from peers, and examples of admired people who have used a growth mindset to succeed.
Given the efforts we have made to craft our interventions, can we expect non-psychologists or teachers to create their own materials and produce positive effects with more informal or in-class activities? Perhaps not, and this brings me to my next point.
We were the ones who sounded the alarm. Although we were originally optimistic about teachers’ ability to readily apply growth mindset in their classrooms, we began to learn things that tempered this optimism. We began to see and accumulate research evidence that the growth mindset concept was poorly understood by many parents and educators and that adults might not know how to pass a growth mindset on to children, even when they reported holding it for themselves. In public essays and published research we began to bring this issue to light. We cautioned that mindsets are not “magic bullets” but depend critically on context and delivery, we argued for more evaluation before scaling mindset programs, and we have argued against including mindset in school accountability systems. When asked to scale our programs across the country, we resisted and said we needed data from our ongoing national study first. We then invited leading researchers in sociology, economics, and statistics to join the efforts, by launching the Student Experience Research Network, which has the explicit goal of understanding how mindset research depends critically on contexts.
More than that, we became deeply committed to learning more about when and how adults can communicate a growth mindset effectively to children and we are researching this vigorously. This is not easy work. A recent survey found that teachers have many misunderstandings about mindset work, and want more resources to communicate mindsets more effectively. Over the next few years, I and other researchers interested in mindsets, along with educators, are taking the responsibility to develop, test, and disseminate detailed material for educators about how to effectively foster a growth mindset in their classrooms. This is all part of the scientific process.
In conclusion, my colleagues and I have been engaged in mindset-related research for many years now, and we have created a body of work we are very proud of. Of course, the house is still being renovated—because science is a process, a long process. However, with collaboration and input from colleagues across many fields, we plan to make a lasting contribution to science and to people’s lives.