Implementation and effects of a school-wide data-based decision making intervention

Abstract

Background.Despite growing international interest in the use of data to improve education, few studies examining the effects on student achievement are yet available.Purpose.In the present study, the effects of a two-year data-based decision making intervention on student achievement growth were investigated.Method.Fifty-three primary schools participated in the project, and student achievement data were collected over the two years before and two years during the intervention.Linear mixed models were used to analyze the differential effect of data use on student achievement. Findings.A positive mean intervention effect was estimated, with an average effect of approximately one extra month of schooling.Furthermore, the results suggest that the intervention especially significantly improved the performances of students in low-SES schools. Interaction EffectsIn Model 5, interaction effects were introduced for school-SES and the intervention, and for student-SES and the intervention.It was hypothesized that the intervention effect would be larger for schools with a large population of students with low-SES (hypothesis 3).The findings of this study support this hypothesis: the interaction effect for school-SES and intervention was positive but not significant for schools with low-SES, and negative but not significant for high-SES schools.Hypothesis 4 concerned the interaction between intervention effect and student-SES.It was stated that the intervention effect would be larger for low-SES students.The interaction effect for intervention and low-SES student was significant and positive (effect of 1.05), but surprisingly the effect found for high-SES students and the intervention was also positive and significant (also an effect of 1.05).Given that the interaction effect for student-SES and intervention is conditional for the interaction effect of school-SES and intervention, it is interesting to compare effects for the combination of student and school-SES even though the latter were not significant.These combinations indicate that the effect of the intervention will only lead to a negative effect on student achievement for medium-SES students in high-SES schools, and is positive and quite large for low-and high-SES students, regardless of their school's average SES.It was assumed that school and student background characteristics would not influence the effect of intervention, and it is interesting to monitor the stability of the estimated intervention effect across models.The estimated positive main intervention effect and the random effect variability over schools is stable across Models 1 to 4. Furthermore, the random intervention effect could not be attributed to differences in background information, or to differential growth in student and school achievement.Therefore, it is concluded that the random intervention effects are identified based on differences in achievement of students in the prior-to-intervention group and during-intervention group.The main intervention effect only decreased after including interaction effects in Model 5.These findings provide partial support for hypothesis 1 (the implementation of DBDM will lead to higher student achievement, there is a positive intervention effect).Although the model fit results are not presented, the multilevel models were investigated with respect to model fit.For each model, a residual analysis was carried out to identify outliers and to investigate distributional assumptions of the residuals.The level-1 residuals showed nine outliers, which most likely stemmed from incorrectly entered observations in the student monitoring system.Some of the scores did not fall within the possible score range of the administered test.Some bias in the fitted values were detected because of observed scores of zero.Since the scores were assumed to be normally distributed and by ignoring this lower-bound, some negative fitted scores were obtained.This lowerbound problem was ignored since the few negative predicted scores did not influence the parameter estimates, which were based on a total of 66,486 observed scores.The student random effects were approximately normally distributed, but the random student effect distribution for grade 6-8 scores was very peaked.The variability in random latent student scores in grade 6-8 was less than expected since the correlation between random effects (grade 3-5 and grade 6-8) was high.The variance in effects across students was relatively low, leading to a more peaked distribution of the random effect distribution of student scores in grade 6-8.However, this random effect was needed to properly represent the scores of students in grade 6 to 8 despite the high correlation with the scores grade 3 to 5. 3The Effects of a Schoolwide Data-Based Decision Making Intervention on Elementary School's Student Achievement Growth for Mathematics and Spelling

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 7 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.