Check the number in the browser, not in the tool
A color fixer said 11 failing pairs before and 2 after. The browser said 57 failing elements before and 10 after. Both were true. A verification pass that reads computed styles is the only number worth reporting, and here is what it found.
Here is a tool that repairs color contrast on a page. You point it at a page, it reads the stylesheet, groups text by the color token it uses, moves the tokens until every pair passes, and writes new CSS. Its summary said: 11 failing pairs before, 2 after. Then a second pass loaded the fixed page in a browser, read the computed color of every text element, and counted again: 57 failing elements before, 10 after. The tool was not lying. It was counting something else.
An automatic contrast fixer of this kind sits between a checker, which only reports, and a designer, who chooses. It is one of the few tools that changes colors on its own, which is exactly why its own account of what it changed cannot be the last word.
What was tested
The subject is a contrast fixer run on a sample page built for the purpose: a mock newsletter page with 76 text elements, 15 color tokens, and 21 distinct text-on-background pairs. Eleven of those pairs failed as shipped. The fixer moves token values with an optimizer and emits a stylesheet override. The verification pass is separate: it loads the fixed page headless, reads getComputedStyle for every text element, resolves its effective background, and computes the WCAG 2 ratio against that element's threshold, 4.5:1 for normal text and 3:1 for large.
| Part | What it reads | What it counts |
|---|---|---|
| The fixer's own view | Tokens and the pairs they form | Pairs |
| The verification pass | Computed styles of every text element on the fixed page | Elements |
Pairs hide elements
The first gap is units. A pair is one row in the fixer's table. An element is one thing on the page a person reads. One pair can cover one element or thirty.
| Failing pair | Elements | Ratio before | Ratio after |
|---|---|---|---|
| Quiet grey on white | 29 | 2.52:1 | 5.41:1 |
| Link blue on navy | 6 | 2.84:1 | 5.00:1 |
| White on gold | 5 | 2.24:1 | 5.25:1 |
| Sky blue on white | 4 | 2.12:1 | 5.39:1 |
| White on sky blue | 4 | 2.12:1 | 5.39:1 |
| Gold on the panel | 2 | 2.07:1 | 4.85:1 |
| Mist grey on the panel | 2 | 1.42:1 | 5.42:1 |
| Quiet grey on the panel | 2 | 2.33:1 | 4.99:1 |
| Gold on white, large text | 1 | 2.24:1 | 5.25:1 |
| White on sky blue, large text | 1 | 2.12:1 | 5.39:1 |
| Pale blue on sky blue | 1 | 1.96:1 | 5.02:1 |
| Total | 57 of 76 |
Eleven pairs, 57 elements. More than half of them are one grey used for every caption and footnote on the page. A summary that says "11 pairs" is correct and tells the reader nothing about how much of the page was unreadable. "57 of 76 elements" does.
The two that stayed failing
The second gap is the one that matters. On the first draft of the sample page, two pairs were still failing after the fix, which the verification pass counted as 10 elements. Both pairs used the same link blue: once on the navy header, where the text had to get lighter, and once on a pale panel, where it had to get darker. The fixer recognised the conflict and reported that it had split the token into two. What it produced was a single new value that served both places. That value settled at 4.40:1 on black and 4.40:1 on the panel, a tenth short on each side.
The fixer's summary said: split 2 conflicting roles and re-optimized. The browser said: 10 elements under 4.5:1. The second statement is the one to report, and the first one, read on its own, would have shipped the page.
Why one value cannot serve both sides is the subject of One brand color cannot serve a light and a dark surface. The fixer's split logic has since been corrected to one private token per background, and the sample page was changed so the conflict no longer appears. On the corrected page the second run went from 57 failing elements to 0 of 76, with 200 optimizer steps in about a hundred seconds.
What the records look like
The fixer's record for one pair, and the verification record for one element in it. The ids are the element indexes the fixer assigned; the selector is the element's path in the page. The link text and the path are lightly generalized; every number is as recorded.
{
"name": "--sky on --paper (normal text)",
"fg_token": "--sky", "bg_token": "--paper",
"threshold": 4.5,
"fg_before": "#7FB8E6", "bg_before": "#FFFFFF",
"fg_after": "#4C6E89", "bg_after": "#FFFFFF",
"cr_before": 2.12, "cr_after": 5.39,
"pass_before": false, "pass_after": true,
"ids": ["0", "57", "58", "..."]
}
{
"id": "6",
"path": "header > div.bar > nav > a",
"text": "News",
"threshold": 4.5,
"fg_before": "#3D6CA3", "bg_before": "#0B2545", "cr_before": 2.84,
"fg_after": "#6D90BA", "bg_after": "#091F3B", "cr_after": 5.0,
"pass_before": false, "pass_after": true,
"fg_token": "--navy-link", "bg_token": "--navy"
}
The pair record is what the tool believes. The element record is what the browser computed after the new stylesheet was applied, including the background the element really sits on. When they disagree, the element record wins, because it is the page a person will see.
Results
| Run | Tool's view, pairs failing | Verified, elements failing | Elements checked |
|---|---|---|---|
| First draft of the sample page | 11 before, 2 after | 57 before, 10 after | 76 |
| Corrected sample page | 11 before, 0 after | 57 before, 0 after | 76 |
The first row is the finding. The second row is the same tool on the same page with the conflict removed, and it shows that when there is no conflict the two views agree at zero. The disagreement is not noise. It appears exactly where the tool's model of the page is simpler than the page.
The fixer does not report a confidence. It reports a loss, which reached zero on both runs. A loss of zero in the tool's own view was consistent with 10 elements failing in the browser. That is the whole point: the tool's objective is not the measurement.
What you can do with this
- When a tool says it fixed contrast, load the result in a browser and read computed styles. Count elements, not pairs, not tokens.
- Report the denominator. "10 failing" means nothing; "10 of 76 text elements" is a number someone can act on.
- When a tool reports that it split a token, check that two values came out. One new value with two jobs is the same problem with a new name.
- Treat a loss, a score, or a pass count from inside a tool as a claim. The verification pass is the evidence. If you build a fixer, make the verification pass part of the output and put its number first.
Limits
- One sample page, built to have the conflict. The 57 and the 10 are facts about that page, not rates to expect elsewhere.
- The verification pass reads the effective background from computed styles. Text over images and gradients is scored against the nearest solid background, which is a simplification.
- Large text was identified by computed font size and weight, not by what a reader would call large.
- A run on a real public page was also made and is not reported here, because its one remaining failure was not investigated.
- The fixer is private. The records above are real; the tool cannot be downloaded from this site.
Try the same discipline on an agent
The agent test corpus on this site works the same way in the other direction. The expected findings are the browser-level truth for 38 small pages. The scorer reports precision and recall per rule and lists every miss. Run your checker on it, then run it on a page your tool has "fixed", and compare what the tool claimed with what the scorer finds.
Sources
- www.w3.org/TR/WCAG22/#contrast-minimum, success criterion 1.4.3, the ratio and thresholds the verification pass applies
- developer.mozilla.org, Window.getComputedStyle, what the verification pass reads
- a11y.world, the agent test corpus and its scorer