How Accurate Are AI Chord Detectors? We Tested at the Piano

Why we bothered doing this

Every chord detection tool, including ours, will tell you it's accurate. Almost none of them tell you how they measured that, or against what. "97.8% string accuracy" sounds impressive until you ask: accuracy on what dataset, judged by whom, and does that number have anything to do with what happens when you point the tool at a real song?

I've been playing for 40 years. I don't trust a percentage I can't trace back to someone actually listening and checking. So instead of quoting a benchmark dataset built by someone else, we sat down at a piano with real songs, real chord charts from two different tools, and worked through them by ear, chord by chord. This article is that methodology and the numbers that came out of it.

The setup

We took a set of songs — a mix of Suno-generated tracks and other recordings — and ran each one through two chord detection pipelines: ours, and the leading MP3-to-chords converter on the market (audio2guitar, the tool most directly comparable to what we do; see the full field in our tools comparison).

For each song, that gave us two independent chord charts. Then the actual work started: I sat at the piano with both charts and the audio playing, and went through section by section, checking two separate things.

First, the key. Is the tonal center right? This is the highest-leverage check, because a wrong key doesn't just cost you one chord, it re-spells the entire song. If a tool calls a song C major when it's really A minor (the same notes, different center of gravity), every single chord name downstream can end up technically defensible but musically wrong. We flagged this before doing anything else, per song.

Second, chord roots at the changes. Wherever the chord actually changes in the recording, does the chart show the right root note? Not the exact voicing or extension necessarily — just: at this moment, is it a G, or is it something else? This is what most people mean when they ask "is the chord detection accurate."

We also tracked a third, harder thing: chord order — whether the sequence of chords in the exported chart actually matches the sequence you hear in the song, in the right places, without chords getting dropped, duplicated, or shuffled by the export step. This turned out to matter more than we expected, and I'll get to why.

The numbers

Piano-verified, against the leading MP3-to-chords converter:

• Chord roots: 91% (mp3chords) vs 80% (competitor)
• Chord order: 79% (mp3chords) vs 34% (competitor)

That second gap is the one worth sitting with. An 80% chord-root score sounds like a B+. But if the chord order comes out at 34%, it means that even when the right chords are technically present somewhere in the chart, they're frequently in the wrong place, or the sections get jumbled during export. A chart like that looks fine at a glance and falls apart the moment you actually try to play along with the recording. Chord order is arguably the harder problem, and it's the one raw "root accuracy" numbers hide.

Why chord order is harder than chord identity

Getting the right root note at a given moment is fundamentally a signal-processing problem: what pitches are present in this window of audio? Getting the order right is a structural problem: does the chart correctly track verse into chorus into bridge, does it avoid collapsing a repeated section into a shorter block, does it avoid duplicating a chord that should only appear once? A tool can nail individual chord identification and still produce a chart that reads top to bottom in the wrong order if its segmentation or export logic is off. That's a separate failure mode from "wrong chord," and in our testing it was the bigger source of unusable charts.

Where things still go wrong (ours included)

I want to be straight about this: 91% and 79% are not 100%. We still miss things, especially on:

• Dense arrangements with a lot of overlapping instruments, where the underlying harmony is genuinely ambiguous even to a trained ear.
• Fast key changes or brief modulations.
• Songs that sit in a minor key with jazzy extensions — these are historically the hardest category for any chord detector, ours included, and it's an area we're actively working on.

If you generate a chart and the first chord sounds off against the recording, check the key first. That's the single highest-value thing to verify, for the same reason it was the highest-leverage check in our own testing.

What this means if you're choosing a tool

Don't just take a headline accuracy number from any tool's marketing page, including ours. Ask what it was measured against, and if you can, verify a chart or two yourself, by ear, the way we did here. If you want to see how this plays out across the five major tools in the category, we cover it in MP3 to chords: the 5 best tools compared. And if you're specifically working with a Suno-generated song, our step-by-step Suno chords guide shows the workflow, including a step where you check the key before trusting the rest of the chart.

Try it yourself

The best test of any chord detector isn't a benchmark, it's your own song and your own ear. Try mp3chords free on your own track and check the chart against the recording the same way we did here.

FAQ

What does "chord root accuracy" actually measure?

It measures whether the chart shows the correct root note at each point where the chord genuinely changes in the recording, verified by a human listening at a piano, not scored automatically against a reference dataset.

What is "chord order" and why does it matter separately from accuracy?

Chord order measures whether the sequence of chords in the exported chart matches the actual sequence in the song, without sections getting dropped, duplicated, or shuffled. A chart can have mostly correct chord identities and still be unusable if the order is wrong, which is why we measure and report it separately.

Why did you compare against audio2guitar specifically?

Because it's the most directly comparable tool to mp3chords in terms of what it attempts (chord and lyric detection from full song audio), and it's a leading option in the category. See our broader comparison in MP3 to chords: the 5 best tools compared for how other tools in the space differ.

Is 91% accuracy good enough to trust completely?

No, and we wouldn't claim that. It means most changes will be right, but dense arrangements, fast key changes, and minor-key jazzy harmony can still trip up any detector, including ours. Always check the key first against the recording before trusting the rest of a chart.

How can I verify chord accuracy on my own songs?

Play the first chord of your chart against the recording and check the key feels right — that's the highest-leverage check. From there, spot-check a few chord changes by ear. mp3chords also lets you edit chords inline if you catch something off, so you don't need to start over.