Pluginval is a plugin dev’s best friend

August 2022 update✨: I teamed up with Dave Rowland from Tracktion and The Audio Programmer to release ValidateMyPlugin.com — the easiest way to get started!

Pluginval is a cross-platform open source tool by Tracktion.

Think of it as quality control for your audio plugin. It loads your plugin as a DAW would, runs myriad tests against it, and reports any problems.

Whether you are having problems with your plugin failing DAW validation or just want to catch issues before your users do, it’s worth learning how to use!

Here are a few tips on working with pluginval:

Pluginval Tip 1: Compile locally

To get started, try validating your plugin on ValidateMyPlugin.com or download and run the latest release from GitHub.

However, to efficiently debug and resolve issues that pluginval unearths, you’ll need to start to running it locally in a debugger.

So build and run the project locally.

Pluginval Tip 2: Run it in CI

You’ll probably want to run pluginval continually while developing a plugin to proactively catch potential problems you are introducing. Otherwise, problems will bunch up and you won’t be quite sure when they were introduced, making resolution more troublesome.

There are some straightforward docs about running in CI, which boils down to sucking down the binary and running it against your built plugin.

You can see a real life config example my Pamplejuce repo which runs pluginval on GitHub Actions.

Pluginval Tip 3: Catch asserts and set breakpoints

It can feel a bit nerve wracking the first few times you run into your first FAIL. What exactly failed? How should you resolve it?

The process to resolve a failure is:

1. Run pluginval from your IDE with a debugger
2. Make sure “Validate in process” is checked so that the debugger can “see through” to your code from pluginval (No longer needed in pluginval 1.0 and higher)
3. Place breakpoints in pluginval itself. For example, search the source for testRunner.setAssertOnFailure (false) and set it to true.
4. Inspect pluginval’s code and the state of your plugin to resolve the issue.

Pluginval Tip 4: Specify a seed to reproduce issues

Pluginval uses randomness throughout the code base (and optionally the test order). This means that you might see intermittent failures and slightly different results on each run.

If you want to reproduce a problematic run, grab the random seed that’s output at the start of the log:

Started validating: AudioUnit-MyPlugin-1682755d-61203125
Random seed: 0x374115a

You can then feed this seed to pluginval as an option, ensuring the exact same set of tests are run. This is especially useful when reproducing an intermittent problem on CI.

Troubleshooting Pluginval: No types found

!!! Test 1 failed: No types found. This usually means the plugin binary is missing or damaged, an incompatible format or that it is an AU that isn't found by macOS so can't be created.

Unfortunately, this error can mean a lot of things.

It boils down to”We couldn’t find your plugin” but there are many reasons for this, especially on macOS.

Verify the following:

1. The plugin is actually built.
2. The path to your plugin is correct.
3. Does your .vst3/.component have spaces in it? (If you are using JUCE, be aware that the name of the plugin is set by PRODUCT_NAME which does allow spaces).
4. Are you trying to validate a .component on macOS? You’ll need to make sure the plugin is picked up by the Audio Unit registry by moving or copying it to /Library/Audio/Plug-ins/components (or the ~/Library version). You might want to run killall -9 AudioComponentRegistrar to kick the registrar to pick it up. Why? Like all JUCE AU hosts, pluginval opens the .plist and gets the component ID out of that to load it up.
5. Are you on macOS? Is it built as a universal binary? Be aware that you can see this error if you have fed pluginval a binary that is on a different architecture than the system you are running on.

I’ve been bitten by all of these problems!

That last issue is more esoteric, but you might encounter it if the plugin you are validating came from a different machine, for example if you are validating plugins in CI. You can check what architectures your plugins support with the file command:

➜  ~ file /Library/Audio/Plug-Ins/Components/ValhallaPlate.component/Contents/MacOS/ValhallaPlate 
/Library/Audio/Plug-Ins/Components/ValhallaPlate.component/Contents/MacOS/ValhallaPlate: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64
- Mach-O 64-bit bundle x86_64] [arm64]
/Library/Audio/Plug-Ins/Components/ValhallaPlate.component/Contents/MacOS/ValhallaPlate (for architecture x86_64):	Mach-O 64-bit bundle x86_64
/Library/Audio/Plug-Ins/Components/ValhallaPlate.component/Contents/MacOS/ValhallaPlate (for architecture arm64):	Mach-O 64-bit bundle arm64

Common Validation FAIL: Not destroying listeners

Pluginval catches all sorts of common pitfalls. Here’s a popular one in JUCE: parameter listeners being referenced after they are destroyed.

This usually boils down to “you forgot to call removeListener in a Component’s destructor”.

It will show up as a memory-related complaint where listeners are called, for example in AudioProcessorValueTreeState.cpp‘s implementation of parameterValueChanged.

Is Pluginval enough?

Occasionally, a plugin will fail validation in a DAW like logic, but pass a strictness level 10 with pluginval. Ideally, those test cases are then identified and contributed back to pluginval itself.

Since version 1.0, pluginval will now run auval for you at strictness levels greater than 5.

On MacOS, you can also apple’s auval tool on the command line to first get more detail on failures.

auval basics

To get more detail than what logic shows you, run auval -v in Terminal.

auval -v TYPE SUBT MANU

TYPE is aufx for an audio effect or aumu for an audio instrument.

SUBT is what JUCE refers to as your 4 character PLUGIN_CODE

MANU is what JUCE refers to as your 4 character PLUGIN_MANUFACTURER_CODE

Here are a couple examples:

aumu Vita Tyte  -  Matt Tytel: Vital
aufx TP3X GDHZ  -  Goodhertz: Ghz Tupe 3

Debugging auval

I’ve never had to deal with this. If you get stuck, here’s a JUCE forum post for you.