Wednesday, April 16, 2008

Digital Sound Separator

New software can modify the individual notes of a recorded chord.

By John Borland

Decomposition: Two views of a digitally recorded chord, provided by Celemony Software's Direct Note Access software. Top: The chord as a single entity. Bottom: The individual notes of the chord extracted. Credit: Celemony Software

Peter Neubäcker, a former German guitar maker turned programmer, has done what many in the computer-music business believed impossible.

A new piece of software called Direct Note Access, first publicly demonstrated by Neubäcker and his company Celemony Software last month, will for the first time allow computers to analyze the digitized sounds of guitar or piano chords, or even multi-instrument recordings, and then extract and modify individual notes.

Computers have revolutionized the recording process, giving sound engineers wide latitude to manipulate notes recorded singly--to change their pitch, their tempo, or where they fall. But teasing apart notes recorded simultaneously, as in a six-string guitar chord, has never before been practical.

"In terms of sound processing, this is kind of the holy grail," says Michael Bierylo, a guitarist and professor of computer music at Boston's Berklee College of Music. "It's something everyone more or less thought we couldn't do."

The new tool promises to give musicians and producers powerful new ways to manipulate recordings both old and new.

It might let studio engineers peer inside a chord-heavy rhythm-guitar part and nudge individual notes into tune. Or it could let them salvage unheard takes by classic musicians like Duke Ellington or Jimi Hendrix, left unreleased due to out-of-tune instruments or misplayed notes. It will certainly give musicians new ability to sculpt sound, such as prerecorded samples or loops, as if they were modeling clay.

Understanding exactly why this is such a breakthrough requires a quick tour of computer music's limitations, however.

Digital audio recording faithfully reproduces the sound of a voice or instrument, just as a tape recorder would. Since the sound is stored digitally, computer tools have been able to stretch, bend, or change it--but only up to a point. Single notes, as in a vocal line, can be easily manipulated. Most big studios today use vocal pitch correctors to repair notes sung out of tune.

But record a guitar or piano playing chords, and things get more complicated. Existing computer recording devices treat these groups of notes as single sonic entities: changing the pitch or timing of one note requires changing the pitch or timing of all the others.

It's that limitation that Neubäcker's Direct Note Access appears to have overcome.

Originally a guitar maker, and with no formal training in physics, math, or programming, Neubäcker got involved in computer music after leading a local lecture group that studied the philosophy and physics of music. He cofounded Celemony in 2000, and a year later, the company released its Melodyne software package, a now widely used tool for analyzing and manipulating single-note audio recordings.

Neubäcker is modest about his own achievements, including the new Direct Note Access tool.

"It's not a very sophisticated approach, with special mathematics, or a special method," he says. "It's more like diligently looking to see what is in there."

"Looking to see" turns out to be what's known as spectrum analysis--graphically or numerically representing a given sound's complex blend of frequencies.

If you pluck the lowest string on a guitar, the sound you hear is dominated by a single fundamental frequency--in this case, an E. But you also hear a set of higher pitches, called harmonics or overtones. (It's the idiosyncratic mix of overtones that makes an E played on the guitar sound different from one played on a trumpet or oboe.)