RL2022-2
Analyzing NAEP and TIMSS Data Using R
Thursday, June 2, 2022
  
INSTRUCTORS
Emmanuel Sikali, U.S. Department of Education, National Center for Education Statistics						

Paul Bailey, American Institutes for Research

Ting Zhang, American Institutes for Research

Michael Lee, American Institutes for Research

Sinan Yavuz, American Institutes for Research

Martin Hooper, American Institutes for Research   
						

This course will introduce the unique design features of the National Assessment of Educational Progress (NAEP) and TIMSS data to researchers and provide guidance in data analysis strategies that they require, including the selection and use of appropriate plausible values, sampling weights, and variance estimation procedures (i.e., jackknife approaches). The course will provide participants with hands-on practice training in analyzing public-use NAEP and TIMSS data files using the R package EdSurvey, which was developed for analyzing national and international large-scale assessment data with complex psychometric and sampling designs. Participants will learn how to perform:

• data process and manipulation,
• descriptive statistics
• cross tabulation and plausible value means, and
• linear and logistic regression

The knowledge and analytic approach learned from this course can be applied to analyzing other large-scale national and international data with plausible values. This course is designed for individuals in government, universities, private sector, and nonprofit organizations who are interested in learning how to analyze large-scale assessment data with plausible values. Participants should have at least basic knowledge of R software (e.g., took an entry level training on R programming) as well as statistical techniques including statistical inference and multiple regression. Having working knowledge of Item Response Theory and sampling theory is preferred. Participants need to have a computer preloaded with the latest version of the R and RStudio software to participate in the hands-on portion.

  
RL2022-3
Advanced Meta-Analysis
Thursday, June 16, 2022
  
INSTRUCTORS
						

Terri Pigott, Georgia State University

Ryan Williams, American Institutes for Research

Tasha Beretvas, The University of Texas at Austin

This course will introduce advanced methods in meta-analysis. Topics covered include models for handling multiple effect sizes per study (dependent effect sizes) and exploring heterogeneity, the use of meta-analysis structural equation modeling (MASEM), and an introduction to single-case experimental design meta-analysis. The statistical package R will be used to conduct the statistical techniques discussed. Participants are encouraged to bring their own research in progress to the workshop. The activities will include lecture, hands-on exercises, and individual consultation. This course is designed to follow the introduction to systematic review and meta-analysis course given by the instructors in prior AERA Professional Development training sessions. The target audience is those researchers with systematic review and meta-analysis experience, but who are interested in learning advanced methods for meta-analysis. Knowledge of basic descriptive statistics, systematic review, and basic meta-analysis is assumed.

  
RL2022-4
Advanced Process Data Analytics using NAEP
Thursday, June 23, 2022
 
INSTRUCTORS
						

Emmanuel Sikali, U.S Department of Education, National Center for Education Statistics

Ruhan Circi, American Institutes for Research

Juanita Hicks, American Institutes for Research

Burhan Ogut, American Institutes for Research

In 2017, the National Assessment of Educational Progress (NAEP) began its official transition to Digitally Based Assessment (DBA) format. The use of DBAs has enabled the recording of students’ interaction with assessment items (e.g., time on task, number of visits, response changes, interactions with graphics or interactive components), as well as with the test interface (use of support functions like drawing, etc.). Course participants will learn how to analyze NAEP Process Data using a process mining framework to understand students’ processes during the assessment. The course is designed for participants interested in using process data. The course is aimed at those with novice to advanced experience working with process data and a solid understanding of coding in R. Attendees will learn about NAEP assessment features, data manipulation and cleaning, sequence formation, which kinds of research questions can be addressed, and the analytic methods used in process mining approaches, specifically a) sequence clustering methods, b) business process mining algorithms, and c) natural language processing approaches.