Live VIdeo Accompaniment

Realtime Multimedia Synchronization for live classical performance

What is LVA

LVA is a plug-and-play software platform that enables classical musicians to deliver precisely synchronized multimedia performances without needing click tracks, manual cues, or a dedicated technician.

Multimedia performance is a powerful way to connect with modern audiences. When visuals are timed to match a musical phrase or dramatic shift, they can reinforce the narrative and phrasing embedded in the music. But tight synchronization in live performance is notoriously difficult to achieve. Traditional fixed-media requires performers to follow a rigid timeline, often via click track, sacrificing flexibility and interpretation. Cueing systems like Max/MSP or QLab offer more freedom, but require specialized operators and often struggle to match musical complexity or speed.

LVA solves this by automatically syncing visuals to live music in real time using machine listening and score alignment. It preserves the spontaneity of live interpretation while delivering cinematic visual impact—ideal for recitals, chamber concerts, staged productions, and educational performances.

How LVA Works

LVA listens to your performance, follows along with the score, and adjusts the video playback in real time—allowing you to stay fully expressive while keeping the visuals aligned with the musical flow.

The system operates in three stages:

  • Listen: LVA captures the live audio as you play, analyzing the sound in real time.

  • Match: It identifies your location in the score using machine learning, tracking your performance moment by moment.

  • Predict & Adjust: At its core, LVA is predictive. Rather than simply reacting to the music, it anticipates how the performance is likely to unfold. Based on what it hears and where you are in the score, LVA continuously updates its estimate of your pacing and adjusts the video accordingly. For example, if you begin to accelerate, LVA will speed up the visuals. If you linger in a ritardando or stretch time for an expressive ornament, like an appoggiatura, LVA will slow the visuals to match.

This adaptive process isn’t about rigid synchronization—it’s about responsive alignment. Think of LVA like an attentive musical partner: it listens, anticipates, and moves with you, allowing your interpretation to guide the performance while keeping visuals tightly connected to the story in the music.

LVA is designed to be intuitive and musician-friendly. To run a performance using LVA, you need two inputs:

  1.  A digital video of the piece: This is a pre-rendered version of the visuals, tightly synced to a recording of the same musical work. It can include animation, live-action footage, generative visuals, or any other visual medium—so long as it’s formatted as a standard video file (e.g., .mp4 or .mov). All of our current productions feature animations by Nikki Pet, but LVA is compatible with any visual style.

  2. The musical score: This allows LVA to “understand” what the performer is playing, functioning much like a digital page-turner. Just as a human page-turner listens and reacts to a pianist’s progress through the score, LVA uses the score to track the performance and synchronize the visuals accordingly.

What You Need to Use LVA

Current Development Stage

The LVA prototype was developed as part of Kaitlin Pet’s PhD research at Indiana University Bloomington, in collaboration with Professor Christopher Raphael. The system builds on Raphael’s Informatics Philharmonic software, originally developed for automatic audio accompaniment—a “musician in a box” system that allows a prerecorded piano or orchestral track to follow a live soloist in real time. Visuals are currently controlled through a Max/MSP patch tailored for responsive video playback. Our current LVA prototype has already been used to produce multimedia performances at venues around the world—including our staging of Stravinsky’s The Soldier’s Tale.

We are now developing a fully integrated, stand-alone version of LVA—specifically tuned for video synchronization—which will be available as a licensed software product. In the meantime, we continue to offer commissions and performances using the current prototype setup.

LVA is currently available by request.

At our current stage of development, we are looking for partners to collaborate in creating multimedia performances leveraging Live Video Accompaniment (LVA). Partnering with us not only brings your artistic vision to life through cutting-edge synchronization tools—it also plays a vital role in advancing the next generation of our platform. If you are interested in working together, contact us.