Audio Reactive Shaders

The soundtrack drives the visuals.

Last updated: 2026-11-20

ALPHA: Shader Workshop is experimental. Audio Reactive can change.

The soundtrack drives the visuals. Add an audio track to a Shader, then let a modulator turn its amplitude or a frequency band into a parameter. MIDI, curve and label tracks drive parameters too. The rest of the controls are on Shader Controls. Rendering is on Shader Output.

Add tracks

Add track sits on the timeline below the time ruler. It opens a menu:

  • Audio file… with the hint "Import stems (kick, bass, vocals…) as separate tracks".
  • MIDI file (.mid).
  • Curve (.csv).
  • Labels (Audacity .txt).

A Shader holds up to 4 audio tracks for now. At the limit, Audio file… reads "Up to 4 audio tracks for now". MIDI, curve and label tracks do not count toward the limit.

Import says "Copying into this Shader and decoding to 48 kHz PCM. The original bytes are kept." Cancel audio import stops it. The audio lives inside the Shader, so it travels with Save As, Looks, history, copies and .hexshader bundles.

Place and trim

An audio lane shows the waveform. "Drag to move. Drag an edge to trim. Use Timing for exact values." The handles are Trim source in and Trim source out. Timing… has Starts on timeline, Source in, Source out and the length in seconds, to at most six decimals. Every move or trim is one Undo step.

The track menu has Create audio modulator…, Create audio set…, Fit timeline to audio, Replace audio…, Timing… and Remove track….

A data lane makes no sound: "Data tracks make no sound: no mute, gain or trims." Its menu has Create modulator…, Timing… and Remove track….

Listening

All audio tracks play together, summed, under the one Shader transport. Scrubbing seeks silently. While a track shows Buffering · playback paused, the visuals pause too. Preparing waveform and playback… shows while a track loads, with Retry if it fails. If playback is unavailable, "Audio playback isn't available on this system. The saved tracks are kept, and Play is refused while they can't be heard."

The clip header has mute and monitor gain. "Mute affects listening only. Visuals and analysis keep running." "Monitor gain changes what you hear only."

Listening on the transport has Monitor latency, This computer only. It "Delays the preview picture to match what you hear (e.g. Bluetooth). Never affects renders. 0–500 ms, default 0." It is not saved with the project. Each modulator's Signal offset (ms) is the authored timing.

Audio modulators

Add modulator on the Modulators card offers Time, Click, Envelope and Audio. Audio appears once the Shader has a track.

For an audio track, Measures is one of two things:

Measure What it reads
Amplitude (RMS) How strong the track is. This is not loudness.
Frequency band (FFT) The energy inside Low (Hz) to High (Hz).

Band presets are Bass (20–250 Hz), Mids (250–2000 Hz) and Highs (2000–16000 Hz). The spectrum picker says "Spectrum at the middle of the track. Drag across it to select a band." FFT bins are about 23.4 Hz apart, so a very narrow band can hold no bin: "No FFT bin center falls in this band."

Gain is 0–100. Attack (ms) and Release (ms) smooth the signal. Presets are Raw (0 / 0), Punchy (10 / 180) and Smooth (80 / 350). Other values show Custom.

Signal offset (ms) shifts the signal in whole milliseconds, −1000 to +1000. A negative value makes the visuals hit before the sound.

Then set the usual Code name, output range and Range or Add mappings. A modulator can also be used by its name in GLSL with no mapping.

Data modulators

  • MIDI: All notes, One pitch or Pitch range (0–127). The value is Trigger, Gate or Velocity.
  • Curve: pick the column, then Linear or Step. Input min and Input max set the input range.
  • Labels: Trigger or Hold, with Label filter (empty reads every label).

MIDI and labels are your markers, not detected beats.

Create audio set

Create audio set… adds four modulators for one lane: A (Amplitude), A1 (Bass), A2 (Mids) and A3 (Highs). The second lane gets B, B1–B3, then C and D. A name already in use takes the next free one, such as A4. One field, Signal offset (ms), all four, offsets the set. One Undo removes the set. "Data tracks take Create modulator, not a set."

A modulator row reads "Audio · · ", or Audio · source unavailable when its track is gone.

Analysis and cache

Analysis runs only when a modulator first needs it. Listening starts none. The card says "Analysing this track once. Later band and response edits reuse it." Cancel analysis stops it, Analyse now starts it, and "Analysis failed." offers Try again. Plan on about 23 MB per minute of audio as an estimate. The result is cached on disk. Clearing the cache never deletes your audio.

Rendering with audio

Every enabled audio signal is frozen into the job when the render is accepted. Preview matches export, and replacing the file or clearing the cache later never changes an accepted render.

The Render dialog shows Ready, Preparing, Change pending, Failed or No curve, and may say "Audio sources are not ready." or "Finish or cancel the audio import before rendering." A render never starts by itself when preparation finishes.

Next to the frames, the render writes soundtrack-<track>.wav for each audio track and timing.json. timing.json has the fps, the exact render start and end, the frame count, and each track's timeline start and source in/out samples. "MIDI, curve and label tracks are baked into the frozen curves and have no soundtrack file." The companions keep true placement. Signal offset (ms) and Monitor latency never shift them.

Not included yet

  • No beat detection. The signal follows the energy.
  • RMS is not loudness.
  • No live input: no microphone, line-in, MIDI, OSC or timecode.
  • No MP4 or video with sound. Use the companion files.
  • No audio authoring through agents or MCP yet.
  • No spectrogram or N-band spectrum.
  • No mixer (solo, pan, effects), clip splitting, stretching or fades.
  • No Ableton or DAWproject import, and no MIDI CC.
Esc
Searching...
No results found.
Type to search the documentation
↑↓ Navigate ↵ Select Esc Close

We use cookies for analytics to understand how visitors use our site. Privacy Policy