Beyond the Digital Curricula Stats: Would This Evidence Hold Up in Front of a School Board?

Every digital learning provider has a shiny statistic to show district leaders. A quote from a superintendent who "loves the results."

Everyone loves a good success story. One that digs into how a district implemented a program, what changed in practice, and what educators learned along the way is genuinely valuable and can offer insights a research study might not.

But that's not usually what shows up in a sales deck. More often, what's presented is a vanity statistic — a number served, a completion rate, or a satisfaction quote, offered up in place of evidence of impact — no comparison group, no independent review, no acknowledgment that it's answering a different question entirely. That's the version worth being skeptical of: not the story itself, but the sleight of hand where an anecdote stands in for proof.

District leaders are often presented with a substitution; an implementation story and a vanity metric being presented as if it were a validated outcome. That distinction matters. Providers aren't necessarily misrepresenting their results; more often, one kind of evidence is being asked to do a job it was never built for, while the questions that actually matter go unasked.

Here's what districts should demand instead, and why so few providers can deliver it.

The Three Questions That Should End Most Sales Conversations

Before a statistic crosses a district leader’s desk, these questions are worth asking:

1. Compared to what? 
   A 90% completion rate means nothing without a comparison group of similar students who didn't use the program. Without one, a provider isn't showing impact. They're showing a number that happened alongside their product, not because of it.
2. Is the study relevant, recent, and related to my situation?
   A good study will describe the district setting, student composition, product version or year studied, methodology, and results. If it doesn’t provide enough information to determine whether the study conditions are similar to yours, it’s probably not rigorous enough to rely on.
3. Who checked the math? 
   Provider-funded, provider-run, provider-reported research isn't independently validated evidence — it's an advertisement with footnotes. The relevant question is who reviewed or authenticated the results. If the answer is "the provider itself," that absence of an independent party willing to put their name on it is telling.

Beyond questioning the individual study, ask: Is this a one-off result, or is there a body of evidence that gives me confidence my district could implement with similar results? A single study, no matter how well designed, is a data point, not a track record. It’s relevant to consider how many independently validated studies exist, and under what conditions they were conducted.

What "Research-Based" Actually Means — and What It's Hiding

"Research-based" is one of the most common claims in this industry, and one of the most loosely used. It's easy to say and rarely checked, so it's worth examining what's actually standing behind the phrase before taking it at face value.

A design claim only holds up if:

• It rests on a true foundational research base — not a handful of cherry-picked studies loosely gestured at, but an actual body of established learning science the design decisions trace back to.
• It's built on a credible logic model that meets the ESSA Tier 4 bar — a clearly articulated, plausible chain from "here's what the research says" to "here's how that shows up in this product," for example, not a vague appeal to "mastery learning" with no visible throughline.
• It's been reviewed by someone outside the company. If the only party that has ever evaluated whether the design logic holds up is the provider itself, "research-based" is merely an assertion, not a substantiated claim.

The Disqualifying Red Flags

District leaders shouldn't need a research degree to protect their districts from vanity statistics or unsubstantiated claims. The following should function as immediate disqualifiers, not minor caveats:

• Self-reported completion or satisfaction offered as the only evidence of impact, with nothing independent behind it. Completing a course isn't the same as learning from it, and a satisfaction score reflects how students felt, not what they gained.
• A success story presented as if it were a research study. A well-done implementation story is worth reading. It's not evidence of impact, and conflating the two doesn't serve districts well.
• No comparison group, ever, in any study claiming to show impact. Without one, "achievement" could mean anything, including simply completing the work.
• A statistic that tries to stand alone. You should be able to access the full research study report, and find information on the district setting, student composition, product version, and year. Otherwise, it’s impossible to tell whether your district could achieve similar results.

There's a wide, meaningful gap between "no independent evidence at all" and "rigorous, current, third-party validated evidence." Providers have gotten very good at living in that gap while sounding like they've cleared it.

What the Bar Should Actually Look Like

Evaluating a program means holding it to a standard built on layers, and identifying who stands behind each one:

• Independent research-based design certification, where an outside organization critically evaluates the research base and verified the product is designed on established learning science, not just adjacent to it.
• Authenticated, independently validated studies on What Works Clearinghouse-eligible outcomes — research confirmed by an outside reviewer of ESSA evidence, not just claimed by the provider. Notable organizations exist specifically to check a provider's tier claim against the actual study design rather than taking the provider's word for it.
• Third-party state and national approvals, where a state agency or national body has reviewed the program against defined standard alignments and quality criteria — not an internal checklist the provider wrote about itself.

That combination — reviewed design, authenticated outcomes, and reviewed approvals — can be hard to come by, precisely because it requires letting outside experts check the work instead of just publishing internal conclusions.

What a Body of Evidence Looks Like When a Provider Can Stand Behind It

Here’s what that looks like in practice, using Edmentum Courseware as an example.

Multiple independent or independently validated studies. Multiple studies across different district contexts are available to districts considering Edmentum Courseware. Study methods were informed by Edmentum’s Technical Advisory Committee, which includes leading researchers from the University of Virginia, University of Cincinnati, and Worchester Polytechnic Institute, and were conducted by research scientists with What Works Clearinghouse certifications or external parties. The full research study reports are available for public download and scrutiny.

• A quasi-experimental study of ninth graders in a large urban Midwest district examined students using Edmentum Courseware for Algebra 1 during the 2021–22 school year, comparing them with students who didn't use the program. Participating students gained up to five percentile points in math on NWEA MAP Growth assessments relative to their peers in traditional Algebra 1 courses — a meaningful result on its own, and one worth taking seriously given how strongly Algebra 1 performance predicts on-time graduation. LearnPlatform by Instructure independently validated the study, confirming it meets ESSA's Level II "Moderate Evidence" standard and the What Works Clearinghouse 5.0 Group Design standard with reservations.
• A study conducted in Clover, South Carolina during the spring 2023 semester looked at students in grades 8–12 using Courseware as part of a structured MTSS program. Students who completed at least one course had 23% fewer absences and 33% fewer tardies than peers who didn't, evidence that the program can move outcomes beyond test scores alone. It's LearnPlatform-validated ESSA Tier 2 research, tracked with a comparison group, not just a completion count.
• A third-party study by Empirical Education examined course success rates for Baltimore City Public Schools students in a summer 2024 credit recovery program.  Students who needed to recover multiple credits and took a mix of traditional and Courseware courses recovered credits at a higher rate than students taking the same number of courses in the traditional setting only. The study also found that Courseware course rigor was comparable, if not even slightly harder. 

Certified research-based design. Beyond the outcome studies, Courseware also holds Digital Promise's Research-Based Design ESSA Tier 4 certification, confirming the product's design traces back to a credible logic model grounded in research on how students learn, not just a marketing claim about it.

Third-party review and approval. Courseware courses are regularly submitted to national and state approval agencies, which evaluate them against a rigorous, published rubric. As of January 31, 2026, Courseware holds 13,500+ active approvals across 50 total lists, for a 99.86% approval rate. Edmentum also holds more course approvals than any other provider on several major lists, including Quality Matters, NCAA, ISTE, University of California a-g, and Indiana MOP.

Customer stories. With rigorous research, certified research-based design, and state and third-party approvals established, the next question is how districts are using Courseware and whether they’re satisfied with it. Courseware has 50+ publicly available customer stories where administrators describe the outcomes they’ve seen in their own districts. 

An evidence base like this is fundamentally different from self-reported numbers and it's the kind of evidence that should be table stakes for a research-based claim, not an exception to it.

Providers should be able to produce their own version of this: multiple outcome studies, including comparison groups, with independent validation, and a design certification confirmed by someone other than themselves. Where that's absent, the gap is itself the answer.

The Real Question to Ask Yourself When Evaluating a Digital Learning Program

Not "does this provider have evidence?" Almost all of them do.

The real question is: would this evidence hold up in front of your school board?

Presented at the podium, in public, with a board member asking the obvious follow-up: What was the research methodology? What outcomes were measured? Who reviewed this besides the company selling it? "One customer said it worked" doesn't survive those questions. "Multiple rigorous, independently validated studies, plus a certified research-base and external state and third-party approvals" does.

Districts deserve the latter.

For Edmentum’s commitments to rigorous research on measurable outcomes and research-based design, visit https://www.edmentum.com/about/our-commitments.