Researchers Urge Caution in Using Measures of Students’
"Non-Cognitive" Skills for Teacher Evaluation, School Accountability,
or Student Diagnosis
WASHINGTON, D.C., May 13, 2015—Policymakers and practitioners have grown increasingly interested in measures of personal qualities other than cognitive ability—including self-control, grit, growth mindset, gratitude, purpose, emotional intelligence, and other beneficial personal qualities—that lead to student success. However, they need to move cautiously before using existing measures to evaluate educators, programs, and policies, or diagnosing children as having “non-cognitive” deficits, according to a review by Angela L. Duckworth and David Scott Yeager published in Educational Researcher, a peer-reviewed journal of the American Educational Research Association.
Personal skills are generally considered to be characteristics that facilitate goal-directed effort, healthy social relationships, and sound judgment and decision making. Basic scientific research reliably shows that measures of these characteristics predict success in school, work, and life. Just a few survey questions, or one session of observing whether a child can delay gratification, can predict educational attainment, income, crime, and happiness months or years later. But these measures are generally not ready for educational use.
“Given the intense visibility and enthusiasm around growth mindset, grit, and other personal skills, it is important for school leaders and policymakers to realize that while there is great benefit to studying and assessing these attributes, the measures should not, currently, be used for broader accountability purposes,” said Duckworth, an associate professor of psychology at the University of Pennsylvania.
“Adapting and using these measures to decide whether a program is working for a school; how to promote or hire or fire teachers, principals, or staff; or how an institution can continually improve its practice and outcomes, are all different from what the measures were developed for,” said Yeager, an assistant professor of psychology at the University of Texas at Austin. “Measures developed for very good research purposes do not necessarily translate into these very important educational purposes.”
Duckworth and Yeager noted that:
In their review, Duckworth and Yeager discussed the limitations and advantages of three common approaches to measuring personal qualities: self-report questionnaires administered to students, questionnaires administered to teachers about their students, and observation of student behavior on performance tasks.
“There really is no perfect measure for any aspect of personal skills. What we have are measures that have their distinct advantages and limitations. Developing better measures, and understanding which currently available measures are appropriate for which uses, are top priorities we should have as an education community,” said Duckworth.
“We advise practitioners and policymakers to seek out the most valid measure for their intended purpose(s),” wrote the authors. “Whenever possible, we recommend using a plurality of measurement approaches. While time and money are never as ample as would be ideal, a multi-method approach to measurement can dramatically increase reliability and validity.”
The authors encouraged further innovation in measurement development, including several “promising approaches” such as creating a suite of web-based, free, scalable “marshmallow test” tasks, or mining students’ online learning behavior or written communication in real time for meaningful patterns of behavior. With the right investments in research and development, Duckworth and Yeager suggested that personal measures could become valuable tools for purposes of program evaluation and educators’ improvement of classroom practices.
The review was made possible by grants to Duckworth from the National Institute on Aging, the Character Lab, the Bill and Melinda Gates Foundation, the Robert Wood Johnson Foundation, the Spencer Foundation, and the John Templeton Foundation, as well as grants to Yeager from the Raikes Foundation, the William T. Grant Foundation, and a fellowship from the Center for Advanced Study in the Behavioral Sciences.
The American Educational Research Association (AERA) is the largest national professional organization devoted to the scientific study of education. Founded in 1916, AERA advances knowledge about education, encourages scholarly inquiry related to education, and promotes the use of research to improve education and serve the public good. Find AERA on Facebook and Twitter.