PEERS Research Methods Series


Visit and Join the PEERS Data Hub: peersdatahub.net

The AERA-ICPSR Partnership for Expanding Education Research in STEM (PEERS) Data Hub announces the PEERS Research Methods Series a webinar series of research capacity-building workshops that focus on research methods used in STEM education research. 

The PEERS Research Methods Series consists of workshops presented in collaboration with the 11 Institutes in Research Methods funded by the National Science Foundation  directed to NSF’s Building Capacity for STEM Education Research (BCSER) program. Aimed at early- and mid-career researchers and scholars, the capacity-building webinar series features projects that seek to promote research expertise in applications of computational, quantitative, qualitative, and evaluative research methods useful to STEM education researchers and broaden the STEM professional workforce.

Upcoming Workshop(s)

 

Past Workshop(s)

Title: Building Big Data Capacity for Education and Social Science Research Communities Using Restricted Administrative Data

Instructor:

  • Jason Owen-Smith, University of Michigan

Date/Time: Thursday, December 9, 2021; 1:oopm-4:oopm (Eastern Time)

Description:

This workshop uses a new administrative dataset about academic research and graduate education – UMETRICS: Universities Measuring the Effects of Research on Innovation, Competitiveness, and Science – to introduce key themes in and methods for “big data” research in education and social science fields.

The “data revolution” has made a wide range of new types of large-scale data such as administrative, social media, textual or electronic data available to researchers in education and social science fields. But unlike more commonly used quantitative datasets such as those derived from surveys, these “found” data were not collected and designed for research.  The workshop faculty will illustrate how “found” data can be used examine topics in STEM education research.

Through lectures and hands-on examples the workshop will focus on:

  • The logic of inquiry for research using large scale administrative data, which differs in important ways from quantitative research using designed data such as surveys
  • Processes and tools for data linkage to integrate information drawn from multiple sources
  • Ethics of working with large scale data with particular attention to privacy and confidentiality
  • Some examples of STEM education research using UMETRICS that will serve as examples of how linked administrative data can inform both current and new lines of research

Graduate students, early career scholars, senior researchers, and others who are interested in using administrative data and new types of large scale data to address STEM education research questions will find this workshop useful. All aspects of the workshop will be accessible to those with little to no technical background in working with large scale data.  Those who are experienced with computational tools and methods will find the value in learning about new data sources for that may be germane to their work.

Title: Getting started in education research with PEER

Instructors: 

  • Eleanor C Sayre, Kansas State University
  • Scott V Franklin, Rochester Institute of Technology

Date/Time: Wednesday, November 17, 2021; 1:00pm–4:00pm (Eastern Time)

Description:

In this interactive workshop, the faculty and participants will work on fundamental skills for research design and collaboration. Discipline-Based Education Research focuses on postsecondary level STEM education through learning sciences and problems of teaching and learning that are grounded within STEM disciplines. Emerging education researchers often need support as they develop their research programs and expand their theoretical and methodological expertise, and they benefit from the guidance of knowledgeable peers and near-peers. Workshop faculty will introduce education research to STEM researchers in the disciplines of biology, mathematics, and physics, as well as those in other disciplines and fields that pursue STEM education studies through basic and applied research.

Using materials from the Professional development for Emerging Education Researchers (PEER) field schools, participants will:

  • articulate research questions and recognize that questions change over time
  • practice key skills for research design, production, and collaboration like generative writing, and playing "yes, and"
  • navigate case studies of complicated, real-world design questions

This workshop is particularly targeted for emerging education researchers, faculty who are new to independent research, and advisors of emerging education researchers. To prepare, think about a research project that you're just getting started or plan to start in the next 6 months. What's exciting about that project? What do you want to investigate? How?

Title: The ProQual Institute for Research Methods

Instructors:

  • Nicola Sochacka, University of Georgia
  • Joachim Walther, University of Georgia

Date/Time: Wednesday, October 13, 2021; 1:00pm–4:00pm (Eastern Time)

Description:

This workshop is for STEM practitioners and researchers across formal and informal K-to-gray settings who want to learn qualitative research methods to broaden their educational change and innovation skillset. The workshop is based on the ProQual Institute for Research Methods, funded by the National Science Foundation (NSF) through its Building Capacity in STEM Education Research (BCSER) program. The workshop aims to build capacity, confidence, and community in STEM education research.

The faculty will introduce a problem-led method to designing qualitative studies (the “Pro” in ProQual), which centers STEM practitioners’ experience and honors their passion and commitment to impact. The approach starts with a pictorial mapping exercise that prompts participants to think deeply about and sketch out the social system surrounding their research interest. The process guides participants to identify the systems’ actors, elements, connections, and structures, with the view to selecting a “Social Reality Under Investigation (SRUI).”

Akin to a “research object” in technical studies, the SRUI delineates the focus of the researcher’s direct empirical efforts and forms the basis for subsequent research design decisions. The resulting map is also a powerful boundary object that can be used to negotiate a shared understanding of the developing project in a research team.

Grounding the research design in the definition of a specific SRUI provides a novel, experience-near way to selecting suitable theoretical frameworks, methods, and methodologies. The workshop faculty will demonstrate this approach through practical examples from real research projects. The workshop will then introduce a research quality framework (the “Qual” in ProQual) as a tool and language to guide the investigation of the SRUI from ideation to propagation.

The ProQual approach will enable STEM practitioners to conceptualize coherent and well-aligned studies that emanate from their specific interests and practice needs. This interactive workshop will allow participants to engage in demonstrations and ask faculty members questions. 

Title:  Maryland Collaborative for Research in Urban STEM Education (MD-CRUSE)

Instructors:

  • Roni Ellington, Morgan State University
  • Vanessa Dodo Seriki, Morgan State University

Date/Time: Wednesday, September 22, 2021; 1:00pm–4:00pm (Eastern Time)

Description:

Outcomes for students in the STEM education pipeline continue are a national concern. Students who learn STEM in urban contexts experience unique challenges that make their trajectory through the STEM pipeline particularly precarious. In this workshop, the faculty provide an exploration of the landscape of urban STEM education and issues that emerge in urban STEM education contexts. Further, the faculty will explore answers to the questions: what is urban STEM education, and what are some of the unique challenges and opportunities available for students who learn STEM in urban contexts? The faculty will also delve into some of the theoretical and methodological perspectives that can inform urban STEM education research with a focus on the types of research questions, data, and methods that can be used to understand issues that arise for students who learn STEM in urban schools. The workshop will end with a discussion of some quantitative and qualitative methods that are used to investigate research questions that are unique to urban STEM education and how these methods can be used to extend our understanding of urban STEM education both locally and nationally. This interactive workshop will allow participants to deepen their understanding of urban STEM education and some cutting-edge approaches to understanding issues unique to the teaching and learning of STEM in urban contexts. 

Title: A LASER Focus on Understanding and Improving STEM Education

Instructors:

  • Shaun B. Kellogg, North Carolina State University​
  • Jennifer Houchins, North Carolina State University
  • Shiyan Jiang, North Carolina State University
  • ​Rob Moore, University of Florida 
  • Joshua M. Rosenberg, University of Tennessee, Knoxville

Date/Time: Wednesday, August 11, 2021; 1:00pm–4:00pm

Description:

This workshop will introduce participants to the Learning Analytics (LA), an emerging research and teaching field sitting at the intersection of Learning (e.g. educational technology, learning and assessment sciences), Analytics (e.g. visualization, computer/data sciences), and Human-Centered Design (e.g. usability, participatory design). LA is proving to be a powerful approach for understanding and improving the digital learning contexts highlighted in the national STEM education plan, while also examining persistent problems in STEM education from new angles. The instructors will provide a brief overview of LA methodologies, literature, applications, and ethical issues as they relate to STEM education, with an emphasis on digital learning environments and broadening participation in STEM programs. Participants will be introduced to open-access curriculum materials developed as part of the Learning Analytics in STEM Education Research (LASER) Institute to gain hands-on experience with computational analysis techniques (e.g. network analysis, text mining, machine learning) using R and RStudio. 

Title: ICQCM: Shaping Critical Data Science for a Diverse World

Instructors:

  • Odis Johnson Jr., Johns Hopkins University
  • Ezekiel Dixon-Roman, University of Pennsylvania
  • Ebony McGee, Vanderbilt University

Date/ Time:  Thursday, July 15, 2021; 1:00pm-4:00pm

Description:

The Institute in Critical Quantitative, Computational, & Mixed Methodologies (ICQCM) aims to advance the presence of scholars of color among those using data science methodologies, and challenge researchers to use those methods in ways that can dismantle the structural barriers to enable human flourishing for underrepresented communities, professionals, and young people. The workshop instructors will discuss and demonstrate how to employ quantitative and computational methods that are situated in context, history, material social relations, and as a product of material and discursive formations. They will explore and provide examples of non-instrumental approaches of quantification that do not essentialize, universalize, or treat data as self-evident, while emphasizing an analytical focus on multiplicity, diversity, and the marginal subject.

Introduction to Qualitative Meta-Synthesis Methods: Achieving STEM Equity and Inclusion through Syntheses
May 20, 2021; 1:00pm -4:00pm ET

Instructors:

  • Maria Ong, Institute for Meta-Synthesis, TERC
  • Nuria Jaumot-Pascual, Institute for Meta-Synthesis, TERC
  • Lisette Torres-Gerald, Institute for Meta-Synthesis, TERC
  • Christina B. Silva, Institute for Meta-Synthesis, TERC

This workshop is for early- and mid-career faculty, researchers, postdoctoral scholars, and graduate students in STEM education and related disciplines to gain basic skills in qualitative meta-synthesis research. Instructors from the Institute for Meta-Synthesis at TERC will present materials they have developed to build capacity in qualitative literature meta-synthesis methods, with a special focus on STEM equity and inclusion literature. They will introduce several aspects of qualitative meta-syntheses research, including: what differentiates literature meta-syntheses from literature reviews; steps for the pre-search process; literature search and selection processes; deductive, inductive, and hybrid coding; and thematic analysis. The instructors will provide demonstrations and hands-on activities for participants, which will be drawn from the instructors’ previous literature meta-synthesis projects focused on the experiences of women of color in STEM (e.g., see paper here: https://onlinelibrary.wiley.com/doi/full/10.1002/jee.20345). By the end of the workshop, participants will have the fundamental skills for conducting a literature search and an understanding of how raw qualitative literature can be coded as data, then transformed into the Findings and Discussion of a meta-synthesis paper.

Mentorship & Collaboration in Quantitative Research: The NSF Quantitative Research Methods Scholars Program

March 17, 2021; 1:00pm -4:00pm ET

Instructors:

  • Laura M. Stapleton, University of Maryland, College Park
  • Gregory R. Hancock, Professor, University of Maryland, College Park
  • Kimberly A. Griffin, University of Maryland, College Park

Early career education researchers often enter the academic and professional community without the training in quantitative research methods that allows them to design rigorous impact and evaluation studies and to be competitive to secure federal grant funding. Team science, a movement toward collaborative research among interdisciplinary research groups (Enhancing the Effectiveness of Team Science, https://www.nap.edu/catalog/19007/enhancing-the-effectiveness-of-team-science), has promise for improving the research success of early career scholars. In this workshop, the instructors present the NSF Quantitative Research Methods Scholars Program, a year-long mentorship and training institute designed after the principles of effective team science that provides quantitative methods training, one-on-one quantitative research mentorship, and facilitated peer-to-peer mentorship for early career researchers.

The instructors will also review strategies for effective peer-collaboration for interdisciplinary research teams, with particular focus on the promises and challenges of collaborations between substantive and quantitative methodological researchers. Workshop instructors will discuss models of mentorship between education researchers and quantitative methodologists and outline ways to construct mentorship relationships to best support the professional development of the mentees. Workshop participants will evaluate their own mentorship and mentee styles and identify the mentorship models most appropriate to their needs. Finally, quantitative mentors and current members of the NSF QRM Scholars Program will share their experiences with the program.

Cutting-Edge Quantitative and Computational Methods for STEM Education Research
February 25, 2021; 1:00pm -3:30pm ET

Instructors:

  • Kenneth Frank, Michigan State University
  • Guanglei Hong, University of Chicago
  • Stephen Raudenbush, University of Chicago
  • Yanyan Sheng, University of Chicago
  • Kaitlin Torphy, Michigan State University

Description:

The overall goal of this webinar is to inform participants of a wide range of significant research questions, data structures, and advanced analytic techniques in the context of theory-driven and data-informed rigorous empirical investigations of STEM education, especially concerning under-represented groups. Cutting-edge methods are essential to study student and teacher experiences with STEM education programs developed, implemented, and evaluated in a complex environment that outstrips what can be rendered by conventional statistical techniques. To illustrate major methodological considerations, instructors will use a stylized case that evaluates the potentially differential impacts of curricular innovations representing the Next Generation Scientific Standards (NGSS) on instructional practices, student engagement, and science achievement. Key methodological issues will be discussed in six inter-related modules:

(1) Design: How to select the sample of schools and teachers and whether to adopt an experimental or a quasi-experimental design suitable for causal inference of the effects of the curricular innovation.
(2) Measurement: How to construct theoretically grounded instruments with strong psychometric properties to measure student engagement, student learning, teacher practices, etc.
(3) Social network analysis: How to represent and model teachers’ interactions with one another as they adapt and implement the new curriculum.
(4) Multilevel modeling: How to represent and model the student, teacher, and school level factors that affect the implementation and outcomes of the curriculum.
(5) Causal mediation analysis: How to examine instructional practices as a mediator of the effects of the curriculum on student outcomes.
(6) Computational methods: How to account for teachers’ and students’ engagement with one another and educational resources on-line.

Each module will consist of a 10-20 minute presentation followed by 5 minutes of Q&A. Some of the instructors will remain after the session for further conversation. Participants are encouraged to visit https://voices.uchicago.edu/nsf-siarm/ for information about a 3-year training initiative led by the team of instructors.

Modern Meta-Analysis Research
February 10, 2021; 1:00pm -3:30pm ET

Instructors:

  • Terri Pigott, Georgia State University
  • Joshua Polanin, American Institutes for Research
  • Ryan Williams, American Institutes for Research

Description:

In this workshop, instructors will introduce participants to basic principles of meta-analysis in the context of systematic reviews for STEM education. The instructors will provide a brief overview of the steps of a systematic review using examples from published STEM education meta-analyses. The workshop will focus on best practice methods for meta-analysis, the synthesis of results from quantitative studies. Instructors will introduce effect sizes typically used in STEM systematic reviews including the standardized mean difference and correlations. Participants will practice extracting information needed for effect sizes from STEM research studies. Instructors will use the R program, metafor, for computing effect sizes. Instructors will also introduce basic meta-analysis techniques for estimating the mean effect size from a set of studies and for exploring effect size heterogeneity across studies. Participants will use metafor to compute the mean effect size and its heterogeneity across a set of studies. The workshop will use small data sets derived from published STEM education meta-analyses to demonstrate the method.


The PEERS Data Hub is joint effort of AERA and the Inter-university Consortium for Political and Social Research (ICPSR) that is NSF funded (Awards ECR 1937612 (AERA) and ECR 1937509 (ICPSR)). It provides collaborative space for STEM education research communities to build and advance knowledge by sharing innovative ideas, methods, and tools. Further information about the PEERS Data Hub is available here.