Repository navigation
Conversation
Problem Pedalboard plugins can't be pickled (spotify#96), so a Pedalboard can't be passed to a process pool, a PyTorch DataLoader with workers, Ray or Dask (spotify#212), saved alongside a dataset, or copied with copy.deepcopy. Solution Add a __reduce__ to Plugin that records the plugin's class, any constructor-only arguments, and the value of every writable property. Unpickling builds a fresh plugin and re-applies those properties. - Pedalboard, Chain and Mix pickle their plugins recursively. - Convolution pickles its impulse response (by filename, or by value if it was passed as an array). This adds a read-only impulse_response_sample_rate property. - VST3Plugin and AudioUnitPlugin pickle their path and raw_state, and are reloaded from that path. This adds a read-only path_to_plugin_file property. - Internal test plugins raise a TypeError instead of losing arguments. Only configuration is pickled, not DSP state: an unpickled plugin behaves as if it was just reset. This is documented in the FAQ. Result All built-in plugins round-trip through pickle with identical parameters and identical output audio, including in a new process. A test also checks that every plugin constructor argument is either a writable property or handled explicitly, so new plugins can't silently lose arguments when pickled.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #96, and makes the parallel processing case in #212 work with process pools.
Plugins and boards can't currently be pickled, so you can't hand a
Pedalboardto aProcessPoolExecutor, a PyTorchDataLoaderwithnum_workers > 0(spawn on macOS/Windows), Ray or Dask, save one alongside a dataset, orcopy.deepcopyit.This follows the approach @psobot sketched in the issue:
Plugin.__reduce__records the class, any constructor-only arguments, and every writable property. Unpickling constructs a fresh plugin and re-applies the properties, so new plugins whose parameters are properties are covered automatically.Pedalboard,ChainandMixpickle their plugins recursively (shared plugins stay shared).Convolutionpickles its IR by filename, or by value if it was passed as an array. Adds a read-onlyimpulse_response_sample_rateproperty for that.VST3Plugin/AudioUnitPluginpickle their path andraw_stateand reload from the path. Adds a read-onlypath_to_plugin_fileproperty.TypeErrorrather than silently dropping arguments.Tests (
tests/test_pickle.py): parameter and output-audio round trips for every built-in plugin with non-default settings, all pickle protocols, nested and empty containers, shared references, both Convolution modes,copy/deepcopy, unpickling in a fresh interpreter, and a guard that fails if any plugin constructor argument is neither a writable property nor handled explicitly.How I tested (macOS arm64, Python 3.12): full
pytest tests(27,130 passed; one timing-based GIL test intest_io.pyfailed under load and passes on rerun),ruff check/ruff format --check, clang-format 14, and pyright on the changed files (0 errors). The external plugin tests could not run locally because loading any VST3/AU aborts in my sandboxed environment (before reaching this code), so those two tests are relying on CI.I used an AI assistant while writing this and reviewed and tested all of it myself.