© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage i Reading and Writing Growth with Vista’s Engage with Literature and Content: A Study of Grades 9-12 Multilingual Learners Click here to learn more about Engage. Click here to contact us.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage i TABLE OF CONTENTS Introduction............................................................................................................ iii Key Findings ........................................................................................................... iv Overview ................................................................................................................. 2 The Role of Language Skills Development ................................................................. 2 Understanding the Learners ..................................................................................... 3 High School Readers and Writers ................................................................................. 4 Implications for Instruction ......................................................................................... 5 Evaluating the Effectiveness of Engage ..................................................................... 6 The Engage Program.................................................................................................... 6 Study Questions ......................................................................................................... 7 How the Study Was Conducted................................................................................. 8 Setting ........................................................................................................................ 9 Timeline ..................................................................................................................... 9 How Reading Growth Was Measured ........................................................................... 9 How We Collected Student Data................................................................................ 10 Participants .............................................................................................................. 10 Initial Reading and Writing Levels............................................................................... 11 Initial Reading and Writing Scores .............................................................................. 11 Student Demographics by Sex and Ethnicity ............................................................... 12 Results .................................................................................................................. 13 Who Completed the Study......................................................................................... 13 Reading Growth Results ............................................................................................ 13 Writing Growth Results.............................................................................................. 15
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage ii Change in Proficiency Levels ..................................................................................... 15 District vs. State Comparisons .................................................................................. 17 Reading Growth by Sex .............................................................................................. 18 Writing Growth by Sex ............................................................................................... 19 Reading Outcomes by Ethnicity ................................................................................. 20 Writing Outcomes by Ethnicity ................................................................................... 22 Context and Considerations ...................................................................................... 23 Visualizing Student Growth........................................................................................ 25 Interpretation of Results ........................................................................................ 26 Understanding the Findings in Context ....................................................................... 26 Summary of Results ............................................................................................... 27 What We Learned................................................................................................... 28 Considerations and Next Steps.................................................................................. 28 Conclusions ............................................................................................................. 28 Evidence Summary ................................................................................................... 29 Reading and Writing Growth from the Beginning to the End of the School Year ............. 29 References ............................................................................................................ 30
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage iii INTRODUCTION SEG Measurement conducted a study to find out how well Vista Higher Learning’s Engage with Literature and Content program helps high school multilingual learners (MLs) improve their English reading and writing skills. The study took place in six Arizona high schools during the 2024–2025 school year and included 171 students in grades 9–12. Students used Engage throughout the year as part of their English language instruction. Their reading and writing growth was measured using the Arizona English Language Learner Assessment (AZELLA) at the beginning and end of the school year. The results showed that students who learned with Engage made clear, measurable progress in both reading and writing. The program worked equally well for boys and girls and for students from di erent ethnic backgrounds, showing that it supports all learners. It was also especially e ective for students who began the year at the lowest proficiency levels, helping them make the greatest gains. This report includes three main parts: • An overview of the Engage program, the study design, and the research questions • A description of the data collected and how the analysis was conducted • The study results, conclusions, and what they mean for classroom practice Together, these findings show that Engage is an e ective, classroom-tested program that helps multilingual learners strengthen their reading and writing skills and build confidence as English users.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage iv KEY FINDINGS Engage has proven classroom impact. Findings show that Engage is a reliable, evidencebased tool that helps multilingual learners build confidence and achieve measurable literacy growth. • Engage works. MLs in grades 9–12 made clear gains in reading and writing after a year of instruction using Engage. • Growth was consistent. Students improved across all grade levels, showing that Engage supports language development throughout high school. • Equitable results were seen. The program was equally e ective for both boys and girls, and for students from di erent ethnic backgrounds. • The biggest gains were made by those who needed them most. Students starting at the lowest English proficiency levels made the strongest progress, especially in writing. In short: Engage gives teachers a practical, research-supported way to help students grow as readers and writers—and to see that growth in action. The findings from this study show that using Engage has a clear and positive impact on high school MLs’ reading and writing skills. Students at all proficiency levels, especially those at academic risk, made meaningful growth after participating in lessons that incorporated Engage.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 2 OVERVIEW SEG Measurement conducted a study of the e ectiveness of Vista’s Engage with Literature and Content for improving the language skills of high school ML students. We found that ML students receiving instruction incorporating Engage achieved substantial growth in reading and writing skills. Engage instruction was equally e ective for both boys and girls and among di erent ethnicities. Engage was also found to be particularly e ective for use with academically at-risk students. The first part of this report introduces Engage and explains the study design, research questions, and data collection process. The second part summarizes the student sample and presents the findings that answer each research question. The final section highlights what we learned from the study, including key results, implications for instruction, and overall conclusions about the e ectiveness of Engage. THE ROLE OF LANGUAGE SKILLS DEVELOPMENT Language skills are essential to academic and career success and vital to personal satisfaction. Language skills have a significant relationship to academic performance throughout students’ schooling. Beyond academic performance, language skills development substantially impacts career opportunities and likelihood of incarceration. While early development of language skills is the broadest challenge, many students continue to struggle with language in adolescence—well into middle school and high school. Research has established that children who have di iculty reading at the end of first grade rarely catch up by the end of elementary school (Torgesen, 2004; Francis et al., 1996; Juel, 1988; Shaywitz et al., 1999; Torgesen & Burgess, 1998). Students who do not master foundational skills early on (by third grade) experience a long-term negative impact on their reading ability and overall academic success. Acquiring reading skills can be particularly challenging for students whose first language is not (MLs). In terms of academic growth, Soland and Sandilos (2020) report that students classified as ML at any point in their academic career had lower achievement than their non-ML peers. One out of every ten students is classified as ML at some point during their K–12 schooling (NCES, 2018). Not only is ML status associated with lower academic expectations from teachers and the students themselves (Kanno & Kangas, 2014), it also contributes to a lower likelihood of taking general and advanced courses in core subjects
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 3 (Estrada, 2014; Umansky, 2016a, 2018). This has contributed to lower academic achievement for MLs (Umansky, 2016b). UNDERSTANDING THE LEARNERS Early intervention is a research-proven e ective measure for reducing—and sometimes altogether preventing—reading disabilities for students at risk of reading failure (Connor et al., 2014; Gersten et al., 2008). However, early intervention is not always successful and as students move into sixth grade and beyond, they are faced with rising text complexity and heavier academic language demands, making literacy development especially challenging for MLs. Reading growth in middle and high school relies on decoding, vocabulary, fluency, comprehension, and disciplinary literacy. Foundational reading skills still matter in secondary school; many students struggle to decode multisyllabic words with automaticity (Wang, Sabatini, O’Reilly, & Weeks, 2019). When word reading is weak, comprehension growth stalls. In one study, word-reading skill accounted for roughly two-fifths of the variance in comprehension gains, with little progress seen below threshold levels (Daniel, Vaughn, Roberts, & Grills, 2022). These findings support targeted word-level work before expecting benefits from higher-order comprehension interventions in secondary grades. Beyond decoding, vocabulary breadth and morphological knowledge are critical to comprehension in adolescence. Reed, Petscher, and Foorman (2016) found that vocabulary accounted for as much as a third of comprehension variance after basic skills were met, though findings are correlational. Evidence for fluency is mixed: A meta-analysis found moderate e ects on oral reading rate (g=0.46), with small transfer to comprehension (Steinle, Stevens, & Vaughn, 2021). Content-embedded language instruction shows promise but uneven transfer; an academic vocabulary course for ML sixth graders produced sizeable gains in taught words (d=0.48) and small comprehension gains (d=0.14) (Lesaux, Kie er, Kelley, & Harris, 2014). Similarly, integrating comprehension with social studies instruction improved content knowledge (d=0.35) and content-specific comprehension (d=0.59), with negligible standardized comprehension growth (d=0.10) (Swanson et al., 2017). In grades 6–7, an extensive decoding fluency program yielded advantages in word accuracy (g=0.32), but not in standardized comprehension, with minimal benefit for students in the lowest vocabulary quartile—underscoring vocabulary as a gatekeeper for MLs (Capin et al., 2024).
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 4 High School Readers and Writers For high school MLs, the stakes are clear. Texts are denser and discipline specific. National indicators show persistent gaps (NCES, 2019, 2022). By secondary school, language factors (Foorman, Petscher, & Herrera, 2018) and core academic language skills (CALS) (Uccelli et al., 2015)—including connectives, complex syntax, and cross-disciplinary discourse—are particularly predictive of comprehension. Students with limited or interrupted formal education (SLIFE) may also face mismatches with US disciplinary text norms (Massachusetts DESE, 2022). Strategic use of students’ full linguistic repertoires can support engagement and understanding, though e ects vary by context and implementation (Hamman-Ortiz et al., 2025). Writing skills share the language development spotlight with reading as secondary students are increasingly called upon to write as part of their learning. Explicit strategy instruction, frequent writing with feedback, and reading-writing integration yield positive average e ects on writing in grades 6–12 (Graham et al., 2023). Impacts are strongest when teachers receive professional development and implement cognitive strategies (Olson et al., 2020; WWC, 2021) supported by explicit instruction in academic language and cohesion (WWC, 2016; Uccelli et al., 2015, Graham et al., 2023). Writing growth is tightly linked to CALS—morphology, syntax/cohesion, connectives, and discourse structures—used across disciplines (Uccelli et al., 2015). High school MLs benefit from genre-specific modeling and argumentation routines (claimevidence-reasoning), supported by explicit instruction in academic language and cohesion (WWC, 2016; Uccelli et al., 2015). Disciplinary writing expectations—e.g., sourcing in history, precise language in math—extend beyond generic forms (Shanahan & Shanahan, 2008). Formative feedback (rubrics, exemplars, targeted comments) can improve argumentative writing, though e ects are typically modest and depend on feedback quality and uptake (Graham et al., 2023).
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 5 Implications for Instruction What does this mean for instruction? Sca olding through modeling, guided practice, and reflection—along with thoughtful first-language use and explicit attention to cohesion— helps multilingual writers build academic writing skills without lowering rigor (WWC, 2016). Because many secondary studies include mixed student groups, results specific to MLs should be interpreted with care (Graham et al., 2023). Screen and remediate decoding until students pass threshold benchmarks; then sustain explicit vocabulary and morphology instruction within content units while embedding brief, text-based comprehension strategy lessons (Wang et al., 2019; Daniel et al., 2022; Reed et al., 2016; Steinle et al., 2022). Make disciplinary language practices explicit (CALS), integrate language and content, and provide structured collaborative routines (Uccelli et al., 2015; Foorman et al., 2018). Given mixed transfer from fluency and variable e ects by entry skills, close fluency gaps strategically and monitor comprehension growth closely. Evidence syntheses with secondary students indicate that explicit vocabulary/morphology teaching and text-based comprehension strategies produce small-to-moderate gains (Capin et al., 2022; Crosson & McKeown, 2019; Sohn, 2023). These strategies, along with appropriate use of the student’s full linguistic repertoire, are likely to meet with greater success in language development (Massachusetts DESE, 2022; Hamman-Ortiz et al., 2025). Despite accumulating evidence, there remain gaps for grades 9–12 MLs: Few studies isolate ML-specific e ects in high school, transfer from word- and fluency-level gains to standardized comprehension is inconsistent, and the role of CALS within content courses needs clearer operationalization and measurement. The continuing need for ML English language instruction at the high school level has led to the development of several products and services to help English learners (ELs) develop language skills. Among those products is Engage, which is the focus of this study.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 6 EVALUATING THE EFFECTIVENESS OF ENGAGE Consistent with ESSA’s evidence standards, this study examines the impact of English language instruction incorporating Engage on grades 9–12 ML student reading and writing outcomes. This study is a story of how high school students can improve their language skills with successful intervention incorporating the program; a story of reading and writing skills development among high school students whose first language is not English. ML students in grades 9–12 classes receiving instruction that included Engage achieved substantial levels of reading and writing skill growth. The Engage Program Engage is a comprehensive literacy program created for multilingual/English learners and striving readers primarily in grades 9–12. The program builds language and reading proficiency through research-driven lessons and activities, engaging texts, and contentdriven lessons. The program provides a variety of print and digital classroom resources that o er support and motivation. Engage provides tools for teachers to assess ongoing learning progress, discover skills gaps, and personalize instruction. Engage is designed to ensure proficiency in vocabulary, reading, listening, speaking, writing, and grammar skills and promote academic language development skills. Engage keeps students’ attention with motivating literary texts and informational texts in science, social studies, math, and the arts and helps them access grade-level content to prepare them for mainstream classes. Engage aligns with the WIDA, CA ELD, TX TEKS, CCSS (ELA and Math), NGSS (Science), NCSS (Social Studies), NCTM (Math), and CEFR (ELT/International) standards.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 7 Study Questions The study posed three questions to guide the collection of evidence in support of the e ectiveness of Engage. 1. Do grade 9–12 ML students receiving instruction incorporating Engage (treatment group) show significant growth in English reading and writing skills measured by the AZELLA (Arizona English Language Learner Assessment) over the course of a school year? What is the magnitude of the growth achieved? 2. Do ML boys and girls in grade 9–12 di er in their growth in English reading and writing skills (reflected on the AZELLA) when receiving instruction incorporating Engage? Do MLs of di erent ethnicity in grade 9–12 di er in their growth in English reading and writing skills (reflected on the AZELLA) when receiving instruction incorporating Engage? 3. Do academically at-risk ML students in grades 9–12 receiving Engage instruction show significant growth in reading and writing skills? How does the growth of at-risk students compare to the growth seen for the full sample of ML students in grades 9–12? The extent to which students receiving instruction incorporating Engage achieve significant growth in English reading and writing skills, and the fact that Engage appears equally e ective for subgroups of the population, are both evidence of e ectiveness.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 8 AZELLA Reading Placement Summer/Fall 24 AZELLA Wri ng Placement Summer/Fall 24 HOW THE STUDY WAS CONDUCTED This study used a pre-post, treatment-only design to evaluate growth in reading and language skills among MLs in grades 9–12. The design compared students’ pre-instruction reading and writing skills with their post-instruction levels to measure growth following instruction that incorporated Engage. To determine Engage’s impact, researchers analyzed overall reading and writing gains and explored di erences by sex, ethnicity, and academic risk status. The following diagram (Figure 1) provides a visual summary of the study’s design and analysis sequence. The extent to which students receiving instruction incorporating Engage demonstrated significant growth in English reading and writing skills, and the fact that Engage appears equally e ective across student subgroups, are both evidence of the program’s overall e ectiveness. Figure 1 Engage Program Study Design (2024–2025) Multilingual Learners Grades 9–12: Pre/Post Growth Model Together, these steps illustrate how the program’s impact on MLs’ reading and writing skills was systematically measured from the start to the end of the school year. Instruc on Using Engage AZELLA Reading Placement Summer/Fall 24 AZELLA Wri ng Placement Summer/Fall 24 AZELLA Reading Assessment Spring ’25 AZELLA Wri ng Assessment Spring ’25
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 9 Setting The study was conducted in six high schools within a school district in Arizona during the 2024–2025 academic year. Participating schools served grades 9–12 MLs enrolled in English language development and literacy intervention courses. The population reflected the district’s linguistic diversity, with students with multiple home languages and varying levels of English proficiency. Timeline The study took place across one full school year, beginning in the fall of 2024 and concluding in the spring of 2025. Students completed pre-assessments at the start of instruction and post-assessments at year’s end using the AZELLA. This allowed researchers to measure both growth in reading and writing skills and progress in proficiency classifications aligned with Arizona benchmarks. The next section outlines how students’ reading and writing growth was measured throughout the school year. How Reading Growth Was Measured The AZELLA was the primary measure used in this study to evaluate MLs’ reading and writing growth. The 2024–2025 results served as the main dataset. Statewide data for that same year were not yet available at the time of analysis; therefore, 2022–2023 statewide data were used for context. AZELLA is administered when students first enter the district and again each winter or spring until they meet the state’s English proficiency benchmarks. Students who achieve proficiency transition into the district’s English Language Arts program and are no longer assessed using AZELLA. According to the Arizona Department of Education (2023), “AZELLA is designed to meet both state and federal requirements to measure Arizona students’ English language proficiency. AZELLA assesses the 2019 Arizona English Language Proficiency Standards (ELPS) and is used for both placement and reassessment purposes for all eligible students in Grades K–12.” The AZELLA demonstrates strong reliability and validity for grades 9–12, with reliability coe icients ranging from .81 to .85 for reading and writing. The test provides both an overall proficiency score and domain-specific subtest scores. This study used reading and writing subtests to measure student growth from pre-instruction (fall 2024) to post-instruction (spring 2025).
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 10 How We Collected Student Data The district provided SEG Measurement with existing student data files for grades 9–12 ML students. These included: • Fall 2024 AZELLA Placement Test test results (pre-instruction baseline) • Spring 2025 AZELLA test results (post-instruction outcomes) Each record contained a unique student ID, ensuring that results could be matched across testing periods. Student background data, including sex and ethnicity, were also included. All files were reviewed for accuracy and completeness before being merged into a single dataset containing one record per student. This combined file was used to analyze reading and writing growth and subgroup outcomes for the study. Participants The study sample included 171 high school MLs (grades 9–12) who had both pre- and postinstruction AZELLA scores available. This approach allowed for valid, “apples-to-apples” comparisons of student progress across the school year. Performance classifications followed Arizona’s benchmark categories—Pre-Emergent, Emergent, Basic, Intermediate, and Proficient. In most analyses, the first three levels were combined into a single Basic category, consistent with state reporting conventions. To provide statewide context, 2023 AZELLA Technical Manual data were used to compare average reading and writing scores by grade, sex, and race. Because 2024–2025 statewide results had not yet been released, the 2023 data served as a suitable proxy for benchmarking. The study sample included 171 high school MLs … who had both pre- and post-instruction AZELLA scores available. This approach allowed for valid, “apples-to-apples” comparisons of student progress across the school year.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 11 Initial Reading and Writing Levels At the start of the study, MLs in grades 9–12 demonstrated beginning English proficiency levels in both reading and writing. The average reading score was 186.24 and the average writing score was 205.97, which fall within the Basic range on the AZELLA scale. A score of 230 or higher is required for classification as Intermediate. Table 1 shows how many students participated at each grade level. Table 1 Grade Level Participation Most participants were ninth graders (54%), typical for newly identified high school MLs. Initial Reading and Writing Scores Table 2 summarizes baseline reading and writing performance before Engage instruction. Table 2 Baseline Reading and Writing (AZELLA Pre-Instruction Scores) Language Skill Area N Mean SD Reading 171 186.24 41.67 Writing 171 205.97 34.24 On average, students began in the Basic proficiency range, indicating a need for continued support in foundational reading and writing. Grade Level N 9 92 10 39 11 21 12 19 Total 171
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 12 Student Demographics by Sex and Ethnicity Of the sample, 57% were boys and 43% were girls. The largest ethnic group was Hispanic (55%), followed by African American (23%) and Asian American (16%). Table 3 provides the sex and ethnic breakdown of the study sample. Table 3 Student Background: Sex and Ethnicity These distributions reflect the district’s diverse ML population and provide context for subgroup analyses reported later. The next section addresses study results, beginning with Study Question 1. Here we examined growth in reading and writing from pre- to post-instruction among the district’s ML students who received instruction incorporating Engage. Background Characteristic N Sex Girls 73 Boys 98 Total 171 Ethnicity African American 39 Asian American 27 Hispanic 94 Other (White [non-Hispanic], multiracial, Native American) 8 Missing 3 Total 171
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 13 RESULTS Who Completed the Study Of the 171 students who took part in the study, 54% were in grade 9, 23% in grade 10, 12% in grade 11, and 11% in grade 12. This mix reflects the typical makeup of multilingual learners in the district’s high school English language development programs. Including students across all four grade levels ensured that the study captured language growth at di erent stages of high school, from newly entering freshmen to students preparing for graduation. Reading Growth Results Question 1: Do grade 9–12 ML students receiving instruction incorporating Engage (treatment group) show significant growth in reading and skills measured by the AZELLA over the course of a school year? What is the magnitude of the growth achieved? To answer this question, we compared pre-instruction and post-instruction AZELLA scores for ML students in grades 9–12 who received Engage instruction. Results are illustrated in Figures 2–3 and Tables 4–7. ML students demonstrated statistically significant growth in reading after one school year of Engage instruction. The average reading score increased from 186.24 (pre-instruction) to 201.67 (post-instruction)—a gain of 15.43 points. This growth was statistically significant (T=9.3; df=171; p< .001) and represented a moderate e ect size (d=.37). Across grade levels, reading e ect sizes ranged from .33 to .47, with the largest gains observed in grade 11 students (d=.47).
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 14 Table 4 reports average reading and writing score changes by grade, showing statistically significant improvement across the board. Table 4 Pre- to Post-Instruction Growth by Grade (AZELLA Reading and Writing) Every grade demonstrated measurable gains; reading e ects were moderate overall. Grade 11 demonstrated the strongest gains. Figure 2 highlights combined reading score gains from fall to spring. Reading scores increased across all grades following Engage instruction. Figure 2: Average Reading Scores Before and After Engage Instruction Skill Area Grade Level N Pre- Instruction Mean Pre- Instruction SD PostInstruction Mean Post- Instruction SD PrePost Mean Di SD Mean di erence (corrected for corr) T Significance Cohens D (corrected for corr) Reading 9 92 179.51 40.99 192.78 36.15 13,27 31.55 4.0 <.001 .34 10 39 179.15 35.25 196.85 41.30 17.69 29.63 3.7 <.001 .45 11 21 196.57 42.47 217.0 43.89 20.43 20.42 4.6 <.001 .47 12 19 221.95 38.33 237.63 49.15 15.68 28.09 4.4 <.001 .33 Total 171 186.24 41.67 201.67 42.25 15.43 42.06 9.3 <.001 .37 Writing 9 92 201.27 36.61 219.42 20,96 18.15 27.02 6.4 <.001 .53 10 39 199.23 31.17 220.36 22.88 21.13 24.12 5.5 <.001 .74 11 21 218.38 29.24 229.24 25.21 10.86 16.64 3.0 <.001 .39 12 19 228.84 18.74 239.63 23.19 10.79 10.65 2.4 =.013 .47 Total 171 205.97 34.24 223.09 22.98 17.12 31.80 6.9 <.001 .54 Pre Instruction; 186,24 Post Instruction; 201,67 180 185 190 195 200 205 Pre Instruction Post Instruction AZELLA Scaled Score Test Timing Pre Instruction Post Instruction
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 15 Grade 9–12 ML students achieved substantial growth in reading skills after receiving English language instruction incorporating Engage. Writing Growth Results Students also showed statistically significant growth in writing. Average writing scores increased from 205.97 (pre) to 223.09 (post)—a gain of 17.12 points (T=6.9; df=171; p< .001; d=.54). By grade, writing e ect sizes ranged from .39 to .74, with the strongest gains in grade 10 (d=.74). Figure 3 shows steady writing growth from before and after using the program. Writing scores improved substantially across grades following Engage instruction. Change in Proficiency Levels Beyond mean score gains, we examined shifts among AZELLA proficiency categories (Basic, Intermediate, Proficient). A di erent view of growth emerges from examining the proficiency classifications achieved by students and the change in classification following instruction. Students are placed into one of three categories describing their level of proficiency based on the AZELLA results. On the AZELLA scale, ranging from 100 to 500, a score of 250 or above is classified as Proficient. Scores between 100 and 229 are classified as Basic; scores between 230 and 249 are categorized as Intermediate. Figure 3: Average Writing Scores Before and After Engage Instruction Pre Instruction; 205,97 Post Instruction; 223,09 200 205 210 215 220 225 Pre Instruction Post Instruction AZELLA Scaled Score Test Timing Pre Instruction Post Instruction
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 16 Table 5 shows how students moved between reading proficiency levels after Engage instruction. Table 5 Reading Proficiency Changes After Instruction To: Basic To: Intermediate To: Proficient From: Basic 124 12 10 From: Intermediate 2 2 8 From: Proficient 1 2 10 About 7% of students initially Basic reached Proficient, and 8% advanced to Intermediate. Among those initially Intermediate, roughly two-thirds attained Proficient, indicating steady reading progress across levels. Table 6 presents movement between writing proficiency levels following use of the program. Table 6 Writing Proficiency Changes After Instruction About 3% of Basic students reached Proficient and 11% moved to Intermediate. Of those initially Intermediate, 64% achieved Proficient, evidence of meaningful writing growth. The story told by the classification data is largely consistent with what we saw for the mean scores, SD, and e ect sizes. The results for both reading and writing paint a picture of growth, albeit modest. For writing (see Table 6), students showed upward movement across proficiency levels after instruction with Engage. About 3% of those who began at the Basic level reached Proficient, and another 11% advanced to Intermediate. Among students who began at the Intermediate level, roughly 64% achieved Proficient status—evidence of meaningful writing growth associated with the program. To: Basic To: Intermediate To: Proficient From Basic 116 15 4 From Intermediate 0 8 14 From Proficient 1 4 9
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 17 District vs. State Comparisons To contextualize the district’s results, we compared proficiency distributions with Arizona statewide AZELLA outcomes (Spring 2023). Table 7 contrasts proficiency levels with statewide percentages by grade. Table 7 District vs. Arizona Statewide Proficiency (Reading and Writing, %) District patterns generally align with statewide results, with notable strengths in grade 12 writing, suggesting Engage supported movement toward proficiency benchmarks. Skill Area Grade Level Statewide N Arizona Statewide Spring 2023 Basic Arizona Statewide Spring 2023 Intermediate Arizona Statewide Spring 2023 Proficient District N District Spring 2025 Basic District Spring 2025 Interme diate District Spring 2025 Proficient Reading 9 7474 68 14 17 92 75 9 16 10 6862 58 16 26 39 83 7 10 11 4985 63 15 22 21 68 14 19 12 3995 58 16 26 19 35 10 55 Total 23,316 64 15 21 171 74 10 17 Writing 9 7474 45 37 19 92 72 19 10 10 6862 41 37 23 39 79 6 15 11 4985 36 38 26 21 57 19 24 12 3995 41 38 21 19 42 20 38 Total 23,316 42 37 21 171 67 15 19
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 18 The results suggest that instruction incorporating Engage is equally e ective for both boys and girls. Question 2a: Do ML boys and girls in grade 9–12 differ in their growth in English reading and writing skills (reflected on the AZELLA) when receiving instruction incorporating Engage? We also examined whether Engage instruction produced di erent outcomes for male and female MLs in grades 9–12. Using Analysis of Covariance (ANCOVA), we compared boys’ and girls’ reading and writing posttest scores while adjusting for di erences in their pretest (initial) AZELLA reading and writing scores. These results are presented in Tables 8–11. The findings show that Engage was equally e ective for both boys and girls in improving reading and writing performance. Di erences between groups were not statistically significant for either reading (F=0.06; df=1/170; p=.81) or writing (F=1.26; df=1/170; p=.26). Reading Growth by Sex Table 8 summarizes the ANCOVA results comparing boys’ and girls’ reading growth after Engage instruction, with initial reading level controlled for. Both groups demonstrated steady language growth consistent with the total student sample. Table 8 ANCOVA Results for Reading Growth (Adjusted for Initial Reading Level) Source Type III Sum of Squares Df Mean Square F Sig Corrected Model 172291.65 2 86145.82 110.37 <.001 Intercept 28519.68 1 28519.68 36.54 <.001 Pre-Instruction Reading 171914.17 1 171914.17 220.25 <.001 Sex 43.90 1 43.90 .06 .813 Error 131132.35 168 780.55 Total 7257899.00 171 Corrected Total 303424.00 170
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 19 The di erence between sexes was not statistically significant, indicating that both boys and girls benefited similarly from Engage instruction in reading. Table 9 provides the adjusted mean reading posttest scores for boys and girls. Table 9 Mean Adjusted AZELLA Reading Posttest Scores by Sex Study Group N Mean SD Girls 73 202.26 44.66 Boys 98 201.23 40.54 Total 171 201.74 42.25 On average, girls and boys demonstrated nearly identical reading scores, confirming the consistent impact of Engage across the sexes. Writing Growth by Sex Table 10 shows the ANCOVA results for writing growth, comparing boys and girls while accounting for initial writing ability. Table 10 ANCOVA Results for Writing Growth (Adjusted for Initial Writing Level) Source Type III Sum of Squares Df Mean Square F Sig Corrected Model 45928.57 2 22964.28 87.94 <.001 Intercept 69924.56 1 69924.56 267.76 <.001 AZELLA PreInstruction Scores 45924.27 1 45924.27 175.85 <.001 Sex 328.16 1 328.16 1.26 .264 Error 43873.12 168 261.15 Total 8600152.00 171 Corrected Total 89801.68 170 As with reading, there were no statistically significant di erences between the sexes, indicating comparable writing progress among male and female students.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 20 Table 11 presents the adjusted mean writing posttest scores by sex. Table 11 Mean AZELLA Writing Posttest Scores (Adjusted for Initial Writing Skills Level) Study Group N Mean SD Girls 73 224.70 25.63 Boys 98 221.89 20.94 Total 171 223.29 22.98 Average posttest scores were nearly identical for boys and girls, reinforcing that Engage supported consistent writing growth across the sexes. Question 2b: Do MLs of different ethnicities in grade 9–12 differ in their growth in English reading and writing skills (reflected on the AZELLA assessment) when receiving instruction incorporating Engage? We also examined whether instruction incorporating Engage was di erentially e ective for students of di erent ethnic backgrounds. Using ANCOVA, we compared reading and writing outcomes across ethnic groups, adjusting for each student’s initial AZELLA reading and writing levels. Reading Outcomes by Ethnicity Table 12 shows the ANCOVA results assessing reading growth from pre- to post-instruction. Instruction incorporating Engage was equally e ective across all ethnic groups in reading, F(4, 171)=1.34, p=.256. This means that after accounting for students’ initial reading levels, there were no statistically significant di erences in reading growth among ethnic groups.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 21 Table 12 ANCOVA Results for Reading Growth (AZELLA Posttest Adjusted for Pre-Instruction Reading Level) Source Type III Sum of Squares Df Mean Square F Sig Corrected Model 176384.90 5 35276.98 45.82 <.001 Intercept 23574.71 1 23574.71 30.62 <.001 AZELLA PreInstruction Scores 157643.01 1 157643.01 204.75 <.001 Ethnicity 4137.15 4 1034.29 1.34 .256 Error 127039.10 165 769.93 Total 7257899.00 171 Corrected Total 303424.00 170 Instruction incorporating Engage was equally e ective across all ethnic groups in reading. After adjusting for students’ initial reading levels, there were no statistically significant di erences in reading growth among groups, as seen in Table 13 below. Table 13 Adjusted Mean AZELLA Reading Posttest Scores by Ethnicity Ethnicity N Mean SD Missing 3 African American 39 195.90 37.57 Asian American 27 198.00 46.39 Hispanic 94 205.94 41.52 Other (White/NonHispanic, multiracial, Native American) 8 205.94 44.14 Total 171 201.67 42.25 Average adjusted posttest reading scores were similar across all groups, with Hispanic students representing the largest subgroup. These findings further support that Engage instruction was equally e ective across ethnicities in promoting reading growth.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 22 Writing Outcomes by Ethnicity We conducted a second ANCOVA to determine whether writing growth di ered by ethnicity, again adjusting for pre-instruction AZELLA writing scores. Table 14 presents these results. Table 14 ANCOVA Results for Writing Growth (AZELLA Posttest Adjusted for Pre-Instruction Writing Level) Source Type III Sum of Squares Df Mean Square F Sig Corrected Model 45982.26 5 9196.45 34.63 Sig. Intercept 58501.66 1 58501.66 220.29 <.001 AZELLA PreInstruction Scores 40441.37 1 40441.37 152.28 <.001 Ethnicity 381.86 4 95.46 .360 .837 Error 43819.43 165 265.57 Total 8600152.00 171 Corrected Total Similar to reading, there were no statistically significant di erences in writing growth among ethnic groups, F(4, 171)=0.36, p=.837. Engage instruction was equally e ective across ethnicities for developing writing skills. Table 15 displays the adjusted mean AZELLA writing posttest scores by ethnicity. Table 15 Adjusted Mean AZELLA Writing Posttest Scores by Ethnicity Ethnicity N Mean SD Missing 3 African American 39 224.13 21.35 Asian American 27 219.72 23.17 Hispanic 94 223.45 22.91 Other (White/NonHispanic, multiracial, Native American) 8 224.46 21.17 Total 171 223.30 22.98
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 23 Mean writing scores were comparable across ethnic groups, again suggesting that Engage supported consistent gains for all students regardless of background. Context and Considerations Because the White/Non-Hispanic, Multiracial, and Native American categories were combined into a single “Other” group due to small sample sizes, these findings should be interpreted with caution. The largest subgroup was Hispanic, which reflects the expected demographics for MLs in the district. While results suggest that Engage promotes equitable growth across ethnic groups, future studies with larger, more balanced samples would strengthen these conclusions. Figure 4 provides an overview of the study design used across all research questions. It illustrates how English reading and writing growth was measured for both at-risk and all ML student groups, from pre-instruction to post-instruction. This visual helps clarify the structure of this study and the sequence of AZELLA assessments used to evaluate growth. Figure 4 Design of the Engage E ectiveness Study (2024–2025): Pre- and Post-Assessment of At-Risk and All ML Students Using Engage With this foundation in place, we next look at how students in di erent grade levels responded to Engage in their reading and writing growth. Instruc on Using Engage AZELLA Reading/Wri ng Placement Summer/Fall ’24 AZELLA Reading/Wri ng Placement Summer/Fall ’24 AZELLA Reading/Wri ng Assessment Spring ’25 AZELLA Reading/Wri ng Assessment Spring ’25 At-Risk MLs At-Risk MLs All MLs All MLs
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 24 Question 3: Do academically at-risk ML students in grades 9–12 receiving Engage instruction show significant growth in reading and writing skills? How does the growth of at-risk students compare to the growth seen for the full sample of ML students in grades 9–12? To address the third research question, we examined the subset of students whose AZELLA scores indicated they were academically at risk—that is, students whose English language proficiency levels placed them at increased risk of academic failure. Students not yet proficient in English face particular challenges as they navigate complex subject-area texts needed to succeed in secondary coursework. AZELLA domain tests are scored on a scale from 100 to 500, with scores of 250 and above reflecting proficiency. Scores between 100 and 229 correspond to the Pre-Emergent, Emergent, or Basic proficiency levels. For this analysis, students scoring below 210 on the AZELLA Reading test (and classified as Basic on the writing assessment) were identified as academically at risk. The score of 210 served as a proxy threshold for risk status in this study. To evaluate whether Engage was particularly beneficial for students performing at the lowest proficiency levels, we compared pre- and post-instruction means for both at-risk and all ML student groups on AZELLA Reading and Writing assessments. Table 16 summarizes these results. Table 16 AZELLA Reading and Writing Growth for At-Risk Students (Grades 9–12) and All ML Students The results show that Engage instruction was highly e ective for academically at-risk students, particularly in writing. Reading gains for at-risk students yielded an e ect size of .56, compared with .37 for the total group. Writing gains were even stronger, with an e ect size of .81 for at-risk students versus .54 for the total group. Skill Area Proficiency Classification N Pre- Instruction Mean Pre- Instruction SD PostInstruction Mean Post- Instruction SD PrePost Mean Di SD Mean di erence (corrected for corr) T Significance Cohens D (corrected for corr) Reading At-Risk ML Students 118 164.95 27.59 179.43 23.50 14.48 28.05 5.6 <.001 .56 All ML Students 171 186.24 41.67 201.67 42.25 15.43 42.06 9.3 <.001 .37 Writing At-Risk ML Students 118 190.94 29.06 210.95 12.93 20.01 26.01 8.1 <.001 .81 All ML Students 171 205.97 34.24 223.09 22.98 17.12 31.80 6.9 <.001 .54
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 25 Visualizing Student Growth To visualize these gains, Figures 5 and 6 display the pre- and post-instruction AZELLA scores in reading and writing for both at-risk and all ML student groups. These figures illustrate the steady upward trend across both groups and the greater relative growth among at-risk students. Figure 5: Comparison of Pre- and Post-Instruction AZELLA Reading Scores for At-Risk and All ML Students Figure 6: Comparison of Pre- and Post-Instruction AZELLA Writing Scores for At-Risk and All ML Students All Students; 0,37 At Risk Students; 0,56 0,3 0,35 0,4 0,45 0,5 0,55 0,6 ALL STUDENTS AT RISK STUDENTS EFFECT SIZE STUDENT SAMPLE All Students At Risk Students All Students; 0,54 At Risk Students; 0,81 0,45 0,5 0,55 0,6 0,65 0,7 0,75 0,8 0,85 ALL STUDENTS AT RISK STUDENTS EFFECT SIZE STUDENT SAMPLE All Students At Risk Students
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 26 Both figures show that while at-risk students started well below the proficiency benchmark, their post-instruction performance displays meaningful progress toward narrowing the gap. INTERPRETATION OF RESULTS While at-risk students demonstrated notable improvement, their post-instruction means (Reading=179; Writing=211) remain below the 250 score proficiency benchmark, suggesting ongoing gaps in language proficiency. Nevertheless, the data indicate that Engage helped these students “catch up” rather than fall further behind. Specifically: • At-risk students’ average reading score (179.43) compared to the total ML group (201.67) shows meaningful progress toward closing the gap. • For writing, at-risk students averaged 210.95 compared to 223.09 for the total ML group. • The e ect-size di erences of +.19 for reading and +.27 for writing are educationally meaningful, suggesting that Engage was particularly e ective for students most in need of language growth. Understanding the Findings in Context Although these results provide encouraging evidence of Engage’s positive impact for at-risk students, interpretation should be made with caution. The study did not include a control group for direct comparison, limiting our ability to determine the exact magnitude of the observed e ects. Even so, the consistent positive direction of the results across both reading and writing suggests that Engage can play a significant role in helping at-risk ML students build foundational literacy skills.
© 2025 Vista Higher Learning Reading and Writing Growth with Vista’s Engage 27 SUMMARY OF RESULTS SEG Measurement conducted a study of the e ectiveness of instruction incorporating Vista Higher Learning’s Engage with Literature and Content program during the 2024–2025 school year. We found that Engage e ectively increased high school (grades 9–12) ML students’ English reading and writing skills. A group of approximately 171 ML students in the district received instruction incorporating Engage and took both a placement test before instruction and a follow-up assessment after instruction. First, we examined the level of reading and writing skills growth achieved by those 171 students to evaluate the level of growth achieved. The extent of reading and writing skills improvement was assessed by comparing the beginning of year reading and writing skills assessment scores to the end of year scores. Students are assessed at entrance to the district and then are assessed every spring until skills are su icient to exit ML instruction. Those students entering the fall of 2024 were measured at the beginning of the 2024–2025 school year and again at the end of the same school year, permitting a sound “apples-to-apples” comparison for evaluating growth. The growth in reading scores from pre-instruction to post-instruction was statistically significant (p<.001), with an e ect size of .37, or about a third of a standard deviation. The growth in writing scores from pre-instruction to post-instruction was also statistically significant (p<.001), with an e ect size of .54, or about a half of a standard deviation. Based on our experience with educational e ectiveness research in authentic settings, .15 to .29 can be considered small, .30 to .49 can be considered moderate, and e ect sizes .50 and above can be considered large. We then turned our attention to the impact of instruction incorporating Engage for population subgroups (Question 2). Specifically, we investigated the impact for boys and girls and among ethnic groups. The results revealed that Engage instruction was equally e ective for boys and girls. Similarly, we found that Engage instruction was equally e ective for all ethnic groups studied. Finally, we examined the growth achieved by at-risk students when receiving instruction incorporating Engage (Question 3). When looking solely at the academically at-risk students, we found that incorporating Engage was particularly e ective in producing reading and writing skills growth. The e ect sizes for at risk students (.56 for reading, .81 for writing) were meaningfully greater than those seen in the total group.
RkJQdWJsaXNoZXIy MjUyNzA0NQ==