Steve Lehman is not the first jazz musician with a Ph.D. in composition—that distinction probably belongs to Mel Powell—nor is he the first to make jazz with the formal and structural tools of the Western classical tradition. He is the first to make it all work on terms that both jazz and classical music can embrace, the first to do things in a way that is both as deep inside the jazz tradition as any “young lion” and as historically and intellectually current as any experimental classical composer. His music ignores the temptations of a retrospective consolidation of the past and instead looks resolutely to the future, with all its unresolved possibilities.

Lehman has a new record, Mise en Abîme, set for release June 24th on Pi Recordings. This is his fifth release on Pi as a bandleader and the second for his Steve Lehman Octet—with Lehman on alto sax, Mark Shim on tenor sax, trumpeter Jonathan Finlayson, trombonist Tim Albright, vibraphonist Chris Dingman, tubist Jose Davila, bassist Drew Gress, and drummer Tyshawn Sorey.

Mise en Abîme marks the third time Lehman has made a conceptual splash on a recording. His first disc for Pi, Demian as Posthuman (2005), challenged the very assumptions of what makes up a piece of jazz music: Is it a tune? A solo? A fragment—such as Dean Benedetti captured out of mid-air and mid-context when he recorded only Charlie Parker’s solos during gigs? Demian is not only fragmentary—the tracks looking into musical ideas in the middle of a thought or a resolution, a riff here, a chunk of solo exposition there—but also has Lehman playing with and against turntables and electronics as much as with musicians. The sidemen—there is no real band—include pianist Vijay Iyer, drummer Sorey, and bassist Meshell Ndegeocello. The music is both aphoristic and dense; nothing sounds like a standard jazz record, but there is a solid core of fundamentals, of working with idiomatic jazz phrasing, articulating an improvisation within the four-dimensional framework of rhythm, pulse, harmony, and time, that is deep within, and deeply reconciled with, the jazz tradition.

Then there is Lehman’s sound as a player, which is a ne plus ultra of jazz aesthetics. His tone is big, cultivated from his studies with Jackie McLean and informed with the acidity and intense physicality of Kenny Garrett and Gary Bartz—as secure as he is in the most complex rhythmic context, Lehman always sounds like he’s leaning into the beat. This gives his playing a very urban funk and swagger, the attitude of a man walking with confidence into every social situation and through every neighborhood.

In 2009, Pi released the first Octet recording, Travail, Transformation, and Flow. Easily the best jazz release of that year and one of the most important records of the new century, this is the great breakthrough for a jazz musician studying at the creative edge of the Western classical tradition. Lehman’s charts for the band combine the wicked grooves of Sorey and the twinned bass instruments of Gress and Davila, with the spectral harmonies pioneered by composers like Gérard Grisey and Tristan Murail—one of Lehman’s instructors at Columbia—in the horns. With a firm, confident grasp, Lehman yanked jazz out of the doldrums of history and into the living present of music making.

Jazz suffers from the burden of a history that its practitioners seem to carry on their backs to every recording session, every gig, every conversation. It’s the history of an American music born from Afro-American origins in a racist country; the history of a popular dance music that, uniquely, transformed itself into a cultish art music; the history that jazz culture tells young musicians they all must work through to “pay their dues” before they are allowed to play with the big boys. What is difficult for jazz lovers is that while this history is substantially true, instead of acting as a foundation it has, in the relatively short period of 100 years, become an anchor that holds the music back from real innovation.

This is a paradox: jazz, since Louis Armstrong and especially since be-bop, has been the improvising soloist’s art, the essence of creativity and seemingly the salient example of musical innovation. But that creativity was based on a rigid idea of musical form and style, and adventurers in form like Stan Kenton, Ornette Coleman, Charles Mingus, Cecil Taylor, Jimmy Giuffre, George Russell, and Miles Davis faced reactionary fear and opprobrium. Armstrong himself denigrated be-bop as “Chinese music,” and Kenton and Taylor still receive a lot of bewildered anger, even from figures who should know better, like writer David Hajdu and musician Branford Marsalis.

I single out Kenton and Taylor to make a point. Jazz made an attempt at formal innovation with the Third Stream movement of the 1950s, the concept of applying technical ideas like counterpoint, complex chords, large scale harmonic structures, and clearly-made forms that are not songs in the conventional sense. The fundamental task was to combine compositional thinking with improvisational opportunities, and the most common result was a lot of intellectually interesting, sonically attractive, but ultimately bloodless music making that ranged from John Lewis to Gunther Schuller. The problem was that jazz, a young music, had a narrow and foreshortened view of classical history. Aside from a bit of Bach, some colors from Debussy, some poorly understood atonality, and the occasional quote from The Rite of Spring that betrayed an utter lack of analysis of Stravinsky’s piece, jazz just didn’t see or hear the long line of continual innovation in classical music.

Kenton, Taylor, Russell, and Giuffre are some of the few who did. By 1957, when Schuller first lectured about the Third Stream, Varèse’s Ionisation was already a quarter-century old, John Cage had produced 4ʹ33ʹʹ, Stockhausen had made Gesang der Jünglinge, and Morton Feldman and Earle Brown had designed important graphic scores. Classical music had advanced exponentially since The Rite of Spring, and the few jazz musicians who sought out contemporary innovations in the music were the ones who were, and still are, least understood.

Moving jazz forward by embracing ideas outside of the blues, song form, head-solos-head arranging methods—not replacing those traditional pillars but adding to them—is a necessity. And Lehman, like some of the misapprehended greats before him, is doing so by adapting cutting edge ideas in classical music, specifically the techniques of spectralism.

Spectralism is a way to mix the possibilities of electronic music with those of acoustic instruments and standard notation. The frequencies that instruments produce on various pitches are analyzed and broken apart. The composer then has those instruments play from the upper frequencies of a fundamental pitch. Put simply, the musicians play notes based on the overtones that are the physical production of sound waves. While the immediate goal of spectralism is to work with timbres, an important result of the technique is to produce harmonies that are anchored on a tonic pitch—as in standard classical and jazz harmony—but that utilize far different tuning than the ubiquitous equal temperament. As the harmonic spectrum expands, intervals fall outside of standard tuning, and some upper partials sound louder than others.

To the ear, the sound has an uncanny brilliance and purity. It feels like the acoustic rug is being pulled away and replaced with something that seems insubstantial but eventually keeps the listener just as firmly supported. In the jazz context of Travail, Transformation, and Flow, the results are revelatory and stunning: exciting rhythms and a brilliant, wide open ensemble sound. Lehman and the other horn players work with microtones in their solos and accompaniment, and the juxtaposition of their spectral organization with the iron-clad equal temperament of Dingman’s vibes produces a completely new dimension in jazz that, while never letting go of the idiomatic roots of the music, is its own, new, expressive quality.

The newness is important. And while a label like “important” conjures up an arid vision of a doctoral student working the software in the music lab, spinning out theoretical charts from a music notation program and mechanically fitting together the pieces, the music is also aesthetically superb.

Lehman is an accomplished jazz musician. He’s as fiercely dedicated to the fundamental, traditional virtues of jazz as any neo-conservative: playing and soloing in time, hitting the downbeat, writing music that is idiomatically jazz in sound and structure. That might seem surprising. With his harmonic experimentation and notated rhythms, Lehman might read as another Lewis or Kenton. But even as he uses those elements and pushes at the edges of song form, he’s entirely a jazz musician and composer. “When you’re writing music for improvising musicians, your number one priority is to create a fruitful space for improvisation. If your interest is in melody, or orchestration, you’re in the wrong idiom,” he told me.

Lehman doesn’t just watch the overtones spread out like ripples in a pond: “I’m kind of walking a fine line between having harmonies based on frequency relationship and having relatively fast harmonic movement, feeling like we really did change to a new color or harmonic root,” he explained.

His compositional chops have grown more fluid and expressive since the previous Octet record, and his use of harmonic rhythm—how the chords move through time and with the music’s pulse—is better, with freer motion and a more powerful pulse. The rhythms are stunning, they grab the body and hold it while the strange sound of the intervals insinuates itself into the ear.

To the original concept, Lehman took the additional step of custom building vibraphone bars that match certain spectral intervals, with the pitches F, A, and C-sharp tuned a quarter-tone sharp. First heard on “Segregated and Sequential,” the sound is jarring, shining an intense light on the unusual harmonies, and the interaction with the horns is much more interesting than on Travail, Transformation, and Flow.

Five years of work and experience also makes for a more assured band. Rhythms that fell on the side of complicated, with an electronic stuttering feel, are now complex, with a constant flow and a clear pulse—or, with Davila playing against Gress, pulses. “The rhythmic structures of the music on the record are all totally written out, and serve as a framework for the drums and bass to expand on,” Lehman said.

The horn players have the rhythmic concept in their bodies and the harmonies in their ears, and cut through the music, including the abstract electronic textures Lehman adds, like shining straight razors:

Everybody has a deeply rooted sense of pulse. Most of those pieces, with maybe one exception, the bar length changes every measure. And a lot of them have meters like 2/4 plus 5/16, things written to imply that things are slowing down, even when they’re not, and asking [the musicians] to feel two pulses at the same time. They know how to do it just because of who they are, and they’re used to how I represent stuff on the page, and verbally.

Those guys, they just know about playing that stuff, they have that legacy of music with a great deal of abstraction but that can be very rhythmically propulsive and groove-oriented. That kind of starts with [Henry] Threadgill and Sam Rivers, and then Steve Coleman and Gary Thomas and Geri Allen, and those guys go on to do their extension of that. All [the Octet musicians] came up with those guys.

Sorey, essential to so much of Lehman’s music and to an enormous chunk of the best contemporary jazz as well, is looser than ever, subdividing less and playing more musically. The whole disc, in fact, is astonishingly musical. If in retrospect Travail, Transformation, and Flow, the best jazz album of 2009, seems didactic, Mise en Abîme is almost frighteningly complex and sublimely rewarding, full of great beauty and a muscular, sophisticated urban attitude and sound familiar from the great hard-bop Blue Note era.

The record builds off and reaches beyond the tradition with three imaginative arrangements of Bud Powell tunes, “Glass Enclosure,” “Autumn Interludes,” and “Parisian Thoroughfare.” In the media material, Lehman writes that:

Bud was one of Jackie [McLean’s] mentors. And like me, Bud was someone who grew up in New York, spent a lot of time in France, and felt a connection to the French classical tradition. For him it was Chopin, for me it’s Tristan Murail and Gérard Grisey.

The disc closes with Lehman improvising poignantly over a tape of Powell at the piano at home.

Making jazz that gets beyond song form, beyond major and minor triads, is:

A big issue yet to be fully addressed in terms of form, cognition, and how reliant we are on harmony to situate us in the form of a piece. If you don’t have a harmonic framework to delineate the form of a composition, you have to find some other way to do it. For me, it’s not easy, people have found different solutions that work to varying degrees, but it’s a trick.

Actually, it’s no trick at all, and Lehman’s dedication to his ideas makes them sound frighteningly easy.