Acoustic lexemes and the TimbreScores

Underneath SoundSpotter is an idea Michael Casey called the acoustic lexeme: a small, repeatable unit of timbre, learned from audio, that behaves a little like a word in a sound language. Cluster a recording into acoustic lexemes and you can describe how its timbre moves over time.

From features to pictures

Once you have that description, you can draw it. In the TimbreScores gallery, each work is rendered as a self-similarity image: moments that sound alike light up against each other, and the large-scale structure of a piece becomes visible at a glance.

The gallery includes Webern's Six Pieces (a study in Klangfarbenmelodie, where the melody is carried by changing timbre), Xenakis's Metastasis, electroacoustic works by Denis Smalley, and Steve Reich's phase pieces. The standard renders use a 40-state model; the transparent-dissimilarity variants apply spherical-projection and affine-shear transforms to the same data.

Why it matters for matching

These pictures are not decoration. They are a sanity check on the features that drive matching: if two moments look similar in a TimbreScore, SoundSpotter is likely to treat them as similar when it searches. Visualizing the model is how you debug the ear.