6 About the i-Ready Assessment The i-Ready reading assessment used in this study is widely used throughout the U.S. The assessment is reported on a vertical scale ranging from 100–800 covering grades K–12; this allows comparability of scores both within and across grades. The reliability of i-Ready is above 0.90. Numerous studies supporting the validity of the i-Ready assessment have been conducted. The i-Ready reading assessment provides an overall score and several subtest scores. The overall score was used for the pretest and posttest for the study. The overall test scores were used since the subtests vary across grade levels limiting the ability to compare results across grades. Study Population and Sampling Creating Matched Samples Random sampling was not possible for this study. SEG Measurement used a matched-groups, quasi-experimental design. For each student in the treatment group, an individual from the control group who was similar in ability and background was selected. We created matched-treatment and control groups using propensity score matching based on multiple factors using logistic regression. We matched students based on the propensity score, a single value reflecting the composite status of each student with respect to the matching variables. For each treatment group student, a comparable student in the comparison group was selected. The treatment and comparison samples were matched based on initial reading skills ability, sex, race, ethnicity, and economic status. The sampling was done without replacement. Matched pairs were required to have propensity scores within 5% of each other. The population of 1773 first, second, and third grade students in the district were included in the matching analysis. We identified 436 treatment-comparison group pairs meeting the criteria, for a total matched sample size of 872 students. Comparability of Treatment and Comparison Groups The initial reading skills levels of the matched treatment and control group were comparable. The difference between the treatment group (mean=441.52) and the comparison group (mean=461.84) was approximately 20 points, or about a third of a standard deviation (SD=63.74). This is within the commonly applied one-half standard deviation guideline for statistical comparison (Schneider, Carnoy, Kilpatrick, Schmidt, and Shavelson, 2007), though modestly greater than the more conservative quarter of a standard deviation guidelines from the U.S. Department of Education (2016). See Table 1. Table 1 Matched Groups Sample Mean Initial Reading Skill Level Study Group Mean n SD Treatment 441.52 436 60.03 Comparison 461.84 436 65.75 Total 451.68 872 63.74
RkJQdWJsaXNoZXIy MjUyNzA0NQ==