How we measure WR3's figures
Every figure we publish about WR3 comes from real recordings of a 2026 focus group wave, compared with a reference made by people. The 90.4% agreement on who is speaking was measured on a block of about two and a half hours, over 141 moments a person had assigned while watching the video, and it includes the human review of doubts. Here we explain what each figure measures, on what sample, and what we have not measured yet.
What the 90.4% agreement on who is speaking measures
The reference. In a real 2026 wave, a person assigned by hand who was speaking at each moment of a block, while watching the video. From that review we took 141 moments, 1 to 28 seconds each, spread over about two hours of the block and spoken by 14 different voices: 556 seconds in all, a little over 9 minutes.
What counts as agreement. For every second of those moments we compare whom WR3 assigns the voice to with whom the person assigned it to. Agreement is the share of those seconds where they match, counted over the seconds in which WR3 detected someone speaking. About 19% of the reference seconds fall in silences between turns or in cuts, and are left out of the count.
What it includes. The 90.4% is the final result, as each person's audio is made: what the machine decides plus the decisions of the person who reviews the doubts in WR3. This is how the figure rises at each stage:
| Stage | Agreement with the reference |
|---|---|
| Comparing microphone levels only | 72.1% |
| Plus voice recognition and the automatic pass, no human decisions | 86.9% |
| Final, with the human review of doubts | 90.4% |
Where the speaker's microphone was synchronized (407 seconds), final agreement is 90.1%. Where it was not (149 seconds), it is 91.6%.
A correction to the reference. The human reference had two people crossed: two participants whose microphones never recorded. We confirmed it with the image of the seats and corrected it before measuring. Against the reference as it was written, final agreement is 82.5%.
The margin. Our target was 90%. The 90.4% clears it by a single moment out of 141, and each moment moves the figure by about 0.7 points. It is the measurement of one block, not an average over many waves.
What a "verified quote" means
A quote passes two checks before it reaches a report.
- Literal check, automatic. When the AI proposes a quote, the system checks that its words appear complete and in order in the transcript of a single turn, ignoring case, accents and punctuation; that the turn belongs to one of the people the idea is attributed to; and that it has at least four words. The text kept is the transcript's, never the one the AI wrote. If it does not match, the quote is rejected.
- Human verification, by listening. A person listens to every quote in the recording and marks it verified or rejected. A finding cannot be approved without at least one verified quote, the report does not publish while an approved finding lacks a verified main quote, and only verified quotes reach the report.
The first check makes sure the quote is in the transcript. The second, that the transcript says what was said in the room. In the real wave, 74 of 78 quotes were verified by listening; the rest did not go into the report.
Where "11 of 120 seconds" comes from
To measure transcription quality we took two-minute windows of a participant's lapel microphone. In one of them, the person wearing that microphone was the loudest voice in only 11 of the 120 seconds. In another window, in 37 of 120. The rest of the time, the microphone was picking up the conversation at the table.
That is why WR3 does not transcribe whole microphone files: it would transcribe the neighbors as if they were the wearer. It first finds each turn and its owner, levels the audio of each turn, and only then transcribes.
What we have not measured yet
- The total time for a full wave, from the arrival of the recordings to delivery. We have measured steps separately, not the whole run. We will measure it on the next wave.
- Human review time per block. Our target is 30 minutes per block and we do not have the measurement yet.
- Agreement on who is speaking across more blocks and more waves. The 90.4% comes from one block. We will publish new figures when we have them.
Behind all these measurements there is one principle: the worst error is the silent one. When WR3 cannot decide who is speaking, it does not guess: it marks a doubt and a person resolves it by listening. In the measured block, where the speaker's microphone was not synchronized, the machine was wrong on 5 moments and flagged 4 of them as doubts, and after review no silent error was left. A doubt is always better than a guess.