Teaching in Educational Administration SIG
Taskforce on Evaluating Educational Leadership Preparation
A joint endeavor of the UCEA executive committee and the TEA-SIG of AERA
Robert Kottkamp and Margaret Terry Orr, co-chairs
Graduate education programs, which dominate the leadership preparation field, need to learn more about how different leadership preparation programs and approaches contribute to the quality and effectiveness of leadership. Moreover, they are under tremendous political pressure to demonstrate their value and impact on the leaders they prepare and the schools that they lead. Yet, leadership preparation programs, despite their prevalence (over 500 nationwide) and importance for school improvement, have rarely been evaluated in a systemic and comparative manner, nor have their long-range impacts been measured.
Two years ago, Robert Kottkamp and Terry Orr invited interested educational administration faculty from around the country to form an informal work group to study these issues and develop a collaborative, comparative evaluation study. Out of this work grew three purposes: (1) to comparatively evaluate leadership preparation programs’ impact on the students served, the schools their graduates lead, and the schools’ educational outcomes; (2) to develop research designs, methods, and instruments that can be replicated and refined through study in multiple institutions and settings to facilitate on-going knowledge development on leadership preparation nationally and internationally; and (3) to engage the leadership preparation field more broadly in the individual and comparative study of their effectiveness and impact.
Collectively, our informal work group (which became formalized as the Taskforce on Evaluating Educational Leadership Preparation) has agreed on four evaluation studies. These are:
- Mapping program designs and prevalence. This proposed study would combine two data collection strategies to map the field of educational leadership preparation. One part would be to try to compile state education agency documentation on the number of certified leaders and educational leadership degree graduates by institution. The second would be to survey programs nationally on their primary content, instructional and organizational attributes, as framed by the national standards for leadership preparation.
- Comparative longitudinal evaluation of programs. A longitudinal, comparative study of public and private leadership preparation programs and their impact on graduate student learning, leadership skills, and associated effects on the schools that they lead and their students’ learning. This is a multi-method study of impacts at the completion of graduate studies and at the school sites where graduates become school leaders. It will combine both cross-sectional studies (looking at the impact on prior graduates) and longitudinal studies (tracking current students over time) from programs of differing assumptions about leadership and its preparation.
- Backward mapping study on leadership effectiveness. This study will look at the preparation roots of effective leadership. By sampling effective schools (schools that defy performance predictions or have made strides in educational improvement in recent years), we will sample and interview principals about the origins of their leadership development.
- Study of the student experience. This study is designed to conduct an in-depth comparison of programs’ admissions and student support practices and their impact on graduate student retention and completion.
Over the past three years, this task force has worked to develop broad professional field support for our research interests and to gain insight into the underlying conceptual and methodological issues. The task force has held half to full day working sessions at the UCEA 2001- 2004 annual meetings, and AERA’s 2002- 2004 annual meetings to explore these issues and develop an evaluation research agenda. Out of our initial work, we developed methodological concept papers, which we presented preliminarily at the UCEA 2002 annual meeting and in revised form at the AERA 2003 annual meeting. These papers are currently being finalized for publication. In addition, our evaluation plans were formally endorsed by the UCEA[1] executive committee and the Teaching Educational Administration Special Interest Group of AERA, as a joint task force endeavor.
The task force is now in the development phase of the research, and seeking funding support. We are beginning to refine our conceptualization, measures, research design, instrument development and research sites’ development. Some of us are conducting mini-studies as pilots for the proposed studies. The University Council for Educational Administration (UCEA) has agreed to serve as the lead institution and use its resources to facilitate communication, research dissemination, and professional development for the field to replicate and build on our work. Its annual conference and publications will be instrumental in our dissemination efforts. The task force will use the TEA-SIG’s annual meeting at AERA for continued presentation of research and methodology.
For further information on the Taskforce and its work, contact Terry Orr (mto10@columbia.edu) or Bob Kottkamp (edarbk@hofstra.edu).
Taskforce Members
Taskforce Related Papers
Orr, M. T., Doolittle, G., Kottkamp, R., Osterman, K., & Silverberg, R. (2004). What are we learning from the ELCC/NCATE required program evaluations? Findings from real-time action research at the UCEA convention. UCEA Review.
Orr, M. T. (2003). Evaluating educational leadership development: What and how to measure leadership preparation and its impact. New York: Teachers College.
Browne-Ferrigno. T & Shoho, A. (2003) Do Admission Processes in Administrator Preparation Programs Assure Students with Potential to Become Effective Principals? Presented at the Annual Meeting of The American Educational Research Association, Chicago, IL.
Orr, M. T. (April 2003). Evaluating educational leadership development: Measuring leadership, its development and its impact. Annual meeting of the American Educational Research Association, Chicago, IL.
Virginia Doolittle, V. & Jacobson, S. with Brenda LeTendre, B. & McCarthy, M. (2003). Tracking leadership preparation to practice and back Presented at the Annual Meeting of The American Educational Research Association, Chicago, IL
Orr, M. T. with Osterman, K. (November 2002). Evaluating educational leadership development: What leadership are preparation program students developing, how do programs facilitate and how can these developments be measured? Annual meeting of the University Council for Educational Administration. Pittsburgh, PA.
Orr, M. T. (Spring 2002). Facing the challenge and continuing the dialogue. TEA-SIG, 9 (1), 1,5
[1] UCEA is formally affiliated (through their leadership taskforce) with the Association of Colleges and Schools of Education in the State Universities and Land Grant Colleges and Affiliated Private Universities whose members include most of the leading leadership preparation programs in the country.