One framework. One measurement language.

How the QERIS engine evolved

The QERIS engine was built in successive stages, each addressing a limit that the previous one had revealed: from proving that automated scanning was feasible, to tying the result to the rule rather than the number of elements, to bringing severity, coverage and confidence into the calculation, and finally to an institutional engine that assesses a full page sample and checks its integrity before any result is issued. Today the QERIS Core Standard 1.0 runs on this engine.

V1 to V4 refer to generations of the scanning engine itself, not to the published methodology version; the methodology and measurement components evolve on separate version tracks, and every result records the versions that produced it.

V1: Proving that scanning works

Starting point

The first question was practical: can public websites be scanned automatically and reliably? Results at this stage were closer to a list of element-level observations on a single test page or a limited scope.

What changed

The engine combined standard automated testing with QERIS-specific rules and, for the first time, produced a result, a score and an initial confidence indicator within a pilot scope.

Result

The idea proved technically viable, and it became clear that counting elements alone does not produce a balanced assessment that can be compared across institutions.

V2: From counting elements to assessing rules

Starting point

The same defect repeated across dozens of elements increased its weight in the score even when it was essentially one defect, moving the number away from the real importance of the issue.

What changed

The assessment started asking whether a rule is met, rather than how many elements failed, so an issue is weighted by its type and impact, while the number of affected elements remains a detail in the report.

Result

A more conservative, more explainable score that is not inflated by a single component repeated across a page.

V3: A mature measurement model

Starting point

The assessment needed to distinguish a critical defect from a minor one, a rule that was actually tested from one with nothing to test, and a confirmed result from one that needs review.

What changed

The model began weighting rules by severity, requiring a minimum of evidence before each aspect is scored, separating passed, failed, not-applicable and incomplete rules, and gradually extending to interaction, language, writing system and text direction.

Result

The score came to reflect the type, severity and sufficiency of the evidence rather than the volume of findings, and the confidence indicator shows how complete the scan was.

V4: An institutional assessment engine

Starting point

Earlier generations assessed a page or a limited scope and were affected by the conditions of a single run, such as a link redirecting to another page or the same page appearing under different addresses.

What changed

Scanning now uses a standard sample of pages with defined roles that represent the institution’s services, verifies the page actually reached and that it belongs to the institution, prevents the same page from being counted twice, and aggregates evidence at institution level after confirming the integrity of the sample.

Result

QERIS moved from a page checker to an institutional assessment engine that is repeatable, reviewable and able to scale to many institutions across countries and languages.

Today: QERIS Core Standard 1.0 on the V4 engine

Starting point

As the registry grew, there was a need to measure what a static check of a page cannot reveal, such as keyboard use and interaction with menus and dialogs, and to keep comparisons fair between websites in different languages and writing systems.

What changed

The assessment now rests on five core standards and 50 rules in which each defect counts once, with real keyboard and interaction checks run safely without submitting forms, a parity layer comparing what is seen with what assistive technology receives, higher evidence requirements for the top results, language-fairness rules and the requirement of an additional language.

Result

A more complete and fairer result, in which the highest grades are backed by sufficient evidence and no institution benefits from something that was not tested.

Before and after: what actually changed?

What distinguishes the current engine?

The value of the current engine does not come from the number of tests alone but from how evidence is managed before the result is calculated: each page must be a genuine part of the sample and within the institution’s scope, repeated pages are not used to inflate coverage, and results are aggregated only after confirming that each page adds independent evidence.

The engine also separates the quality of the website itself from temporary technical failures such as blocked access or slow loading, so a temporary problem does not become a judgement on the institution, and the previously published result stays in place until a new scan completes successfully.

Interaction checks are reviewed continuously on real websites to make sure they detect actual errors without false results, and institutions are scanned again after every material update so that results remain comparable.

How are versions tracked?

Every published result records the methodology and measurement-component versions that produced it, so it can be read in its proper context. When a material change is adopted it receives an explicit version, and institutions are scanned again so that all of them are compared under the same version.

Summary

In short: V1 proved scanning was possible, V2 fixed how issues are counted, V3 matured the measurement and confidence model, V4 turned the engine into an institutional assessment tool focused on sample integrity and consistent evidence, and the Core Standard 1.0 now adds broader keyboard and interaction measurement and clearer fairness across languages.