Mindsets and the learning environment: Assessing the African American Male Achievement Program in Oakland Unified School District
The Student Experience Research Network launched an interdisciplinary initiative in Fall 2016 to explore how learning environments shape the mindsets students develop about learning and school. The project’s aim is to generate scientific evidence about how educators, school systems, and structures can convey messages to students that they belong and are valued at school, that their intellectual abilities can be developed, and that what they are doing in school matters.
Fourteen projects were awarded over two rounds of the initiative. Funding was generously provided by the Bill & Melinda Gates Foundation, Joyce Foundation, Overdeck Family Foundation, and Raikes Foundation. Seventeen different Network scholars are participating along with over 20 external collaborators. The projects span a wide range of topics, from exploring how teacher practices cultivate learning mindsets and identity safety in K-12 classrooms, to the relationships between learning mindsets and neural processes throughout adolescent development.
This is part of a series of blog posts in which we will hear from the leaders of each of the projects funded in the second round of the initiative to find out more about the questions they are exploring, what they are learning, and how their work is advancing the field of mindset science.
The next project in our Mindsets and the Learning Environment portfolio, The Effects of the African American Male Achievement Program, evaluates the African American Male Achievement (AAMA) program, the first program in the nation to embed a culturally-centered curriculum that is specifically targeted to black male students, who face systemic barriers to academic success. AAMA launched in 2010 and is part of the Office of Equity within Oakland Unified School District (OUSD). The program is an example of “targeted universalism,” a concept developed by john powell that recognizes universal goals for all students, but also emphasizes the need for strategies that meet the unique needs of groups who have inequitable access to opportunities and distinctive cultural contexts.
This study builds on a 2014 report by Vajra Watson about AAMA and is the first independent quantitative examination of the program’s effects on academic outcomes, including attendance, grades, standardized test scores, and disciplinary actions.
Who are the members of the research team?
Mindset Scholar Thomas S. Dee and Emily Penner, an assistant professor of education at the University of California, Irvine, are leading the research project in coordination with the John W. Gardner Center for Youth and Their Communities at the Stanford Graduate School of Education.
What is AAMA?
Two of the primary goals of AAMA are to foster and reinforce black male students’ positive identity formation and sense of belonging in school. The program provides comprehensive academic mentoring that takes place regularly as part of the school day, rather than episodically in an extracurricular setting. AAMA also works with district leadership to provide professional development for teachers and engages parents to support their child’s college readiness.
The centerpiece of AAMA is a set of elective courses led by black male instructors who are carefully selected based on cultural competency, understanding of youth development, and teaching experience. The courses emphasize broad academic mentoring, including leadership and character development activities, personalized support (e.g., transcript evaluation and guidance counseling), and field trips that expose students to colleges and careers. The courses use a culturally relevant curriculum and pedagogy, relying on materials and instructional methods that align with the experiences of the students. Finally, the courses promote peer-support by emphasizing unity among all students and by clustering students with diverse achievement levels in the same classrooms. Outside of school, the program also features conferences, community gatherings, and a summer internship program.
What is the purpose of the project and how will it fit into the field of mindset science?
Research in the social sciences has found that even brief psychological interventions that target important psychological questions and concerns can have powerful effects. Sustained initiatives like AAMA, which is embedded in multiple school activities and is responsive to the specific psychological and cultural experience of black male students, offer opportunities to extend our understanding of more comprehensive approaches that attend to students’ psychological experience of school.
In their previous work, Tom and Emily found that participation in an ethnic studies course offered by the San Francisco Unified School District substantially improved attendance, grade point average (GPA), and credit completion among 9th grade students. Before that study, Emily explains, “although there was growing enthusiasm and largely qualitative evidence to support the efficacy of ethnic studies courses, there was no strong causal evidence. In that sense the large, positive effects we found were surprising. But, given that this course functioned like a year-long social-psychological intervention, maybe it’s a bit less surprising.”
“Our results led us to consider whether other programs with a strong commitment to racial equity that valued students’ racial identities and histories while buttressing their academics could also have similar impacts,” said Emily. “That really led to our interest in the AAMA program.” The AAMA adds on key features, such as mentorship and peer-to-peer support networks, to the culturally relevant pedagogy studied in San Francisco.
The evidence provided by this project will inform refinements to AAMA and will provide similar programs (e.g., President Obama’s My Brother’s Keeper initiative) and the field at-large with a valuable review of the challenges and opportunities that exist in sustained initiatives like this one. The researchers hope their assessment of AAMA can serve as a “proof point” that stimulates the design, adoption, and implementation of programs for other groups that have experienced inequitable access to opportunities in education contexts.
About the data
Through the Youth Data Archive maintained by the Gardner Center at Stanford, the team has access to longitudinal student-level data from OUSD spanning the periods before and after the program began, from academic year 2008-2009 to 2016-2017. They will use transcripts to determine which students participated in AAMA. The study will investigate the effect of participating in AAMA on several measures of students’ academic engagement (e.g., chronic absenteeism and the probability of being suspended or expelled) and academic outcomes (e.g., grades in core academic subjects, test performance, school dropout, and college matriculation).
Year one progress
This project is on a two-year timeline. The first year has been focused on data collection and cleaning. While the researchers have not begun their core analyses, they have culled important lessons about large-scale data collection within a school district.
The project has greatly benefitted from key partnerships with the Raikes Foundation and the Gardner Center at Stanford and, in turn, their long-standing relationships with OUSD. With these partnerships in place well before the project started, all stakeholders had a shared sense of purpose and a shared approach to collecting and using data.
For similar projects, the researchers recommend data systems that focus on valid measures and the integration of different data fields both at a moment in time and for students over time. They also emphasize the importance of documenting school and district initiatives with a level of rigor that can support formal analyses.
This project makes a compelling case for investments in district-level data systems and highlights the value of dialogue between researchers, practitioners, and trusted intermediaries in the notoriously challenging process of managing longitudinal student data.
What are the next steps for the project?
The project team will analyze the causal impact of AAMA on black male students’ academic outcomes using two strategies. First, the team will compare the changes observed among AAMA participants to the changes observed during the same time frame among students who did not participate using a rigorous statistical method that researchers call a “difference in differences” design. Second, the team will leverage the staggered roll-out of AAMA at different schools in Oakland to compare changes in student outcomes at “treatment” schools to changes at “control” schools.
“We plan to look at impacts on not only academics,” Emily explains, “but also behavioral indicators that likely pick up on school belonging, such as attendance and dropouts.” Together, these data could help unpack the impact of the AAMA program for students who participate, as well as the impact of any “spillover” effects for non-participating students. The Student Experience Research Network will continue to publicize this important work as it progresses.