Here is another take on the extensions API – I am already supporting this in Rhino, so I am confident it is not too difficult to implement this in a plugin.

As explained before, Zen-aware plugins inherit from ZenAwareAudioFX instead of AudioEffectX. They can then exchange information about the current preset with the host, provided they implement the methods described below.
For the sake of simplicity, there are 3 different logical groups of extensions:

Presets comments

Many plugins have a text zone providing textual information about the preset, for instance playing tips. It would be useful to retrieve this information in Zen, so that full-text search can apply to the preset comments as well. The comments are exposed to the host through the following methods:

virtual VstInt32 zenGetPresetCommentsSize() const;
virtual void zenGetPresetComments(char* comment) const;
The host first calls zenGetPresetCommentsSize to ask the size of the comment area. It then allocates memory accordingly (to store the comments) and calls zenGetPresetComments, which writes the comments into the provided area. If the comment spans over several lines, they are separated by a \r (0×0A) character.

User control assignments

A lot of plugins provide preset-dependent “performance” parameters, that allow to quickly access the most useful controls of a preset. The following 2 methods are used to export these assignments to the host.

virtual VstInt32 zenGetNumberOfUserControls() const;
virtual VstInt32 zenGetDefaultUserControlAssignment(unsigned int ctrlIdx) const;
The host will first call zenGetNumberOfUserControls to ask how many performance parameters the plugin provides. Then it will call zenGetDefaultUserControlAssignment for each performance parameter. This last method returns the ID of the VST parameter associated to the performance control.

External files (aka “assets”)

These extensions are really useful for plugins relying on external files, such as samples. Remember, Zen distributes presets to many users, so if a preset requires a particular sample file, we need a way to somehow “attach” the sample to the preset, and distribute the sample too. This is the role of the following “assets” extensions.

Before going into the details of the extensions, here are some general remarks about assets:

• Every plugin has its own way to identify assets. Zen uses a char[256] to store asset IDs in a plugin-independent form. So if your plugin uses numeric IDs for assets, you can use itoa() or equivalent to fill in the asset name.
• Imagine a plugin playing large samples, using disk streaming. It would not be efficient to send these samples to the host by loading them entirely into memory. So Zen uses a “chunks” mechanism to exchange assets, where large data is split into smaller parts. The maximum chunk size is 1 Mb. So if you use a 2.5Mb sample, Zen will ask you for 3 chunks. You will be free to split your data in any way you want, as long as no part is bigger than 1Mb.

virtual unsigned int zenGetNumberOfAssets() const ;This method returns the number of assets required for the current preset.

virtual Zen::ZenErrorCode zenGetAssetInformation(unsigned int assetIdx, Zen::ZenAssetInformation* assetInfo) const;When Zen calls this method, the plugin fills an assetInfo structure with information about the nth asset in the current preset:
struct {
const char *assetName;
unsigned int totalSize;
unsigned int nChunks;
bool dontPersist;
} ZenAssetInformation;

• assetName is used to identify the asset.
• totalSize is the total size of the asset (in our example above, 2.5M).
• nChunks is the number of chunks the plugin wants to use (in our example, it has to be 3 or more).
• Finally, you can ignore the dontPersist flag for now

virtual Zen::ZenErrorCode zenGetAssetDataChunkStart(unsigned int assetIdx);
virtual Zen::ZenErrorCode zenGetAssetDataChunk(unsigned int assetIdx, Zen::ZenAssetDataChunk* chunkData);
virtual Zen::ZenErrorCode zenGetAssetDataChunkEnd(unsigned int assetIdx);

Zen will call these 3 methods in sequence, for each chunk. zenGetAssetDataChunkStart and zenGetAssetDataChunkEnd are convenience methods, in case the plugin would need to do something before (and after) the chunk is queried. zenGetAssetDataChunk is where the plugin gives the chunk data to the host., using this structure:
struct {
const char* assetName;
unsigned int chunkIndex;
unsigned int chunkSize;
void* chunkData;
} ZenAssetDataChunk;
Zen calls zenGetAssetDataChunk with the chunkIndex value set to the number of the chunk it wants. The plugin simply needs to fill the remaining members.

So, to summarize: regarding assets, your plugin needs to implement the following 3 methods:

• zenGetNumberOfAssets() to tell the host how many assets are required for the current preset.
• zenGetAssetInformation() to give details about the nth asset.
• zenGetAssetDataChunk() (and optionally, the associated begin() and end() methods) to fill a chunk of a given asset.

Once this is done, ZenAwareAudioFX gives you nice little methods that you can use to retrieve your assets from the host:

typedef void (*ZenEnumerateCallback)(const char* assetName, void* userData);
Zen::ZenErrorCode zenEnumerateAssets(Zen::ZenEnumerateCallback callback, void* userData);Is used to list all the assets that Zen manages for your plugin. Zen will call the provided callback, once per asset, with the assetName.

Zen::ZenErrorCode zenQueryAssetInformation(Zen::ZenAssetInformation *info);This method asks the host about an asset identified by its name. The ZenAssetInformation structure is the same as above, but this time:

• You fill in the assetName member to tell the host which asset you are interested in
• Zen replies with information about the asset (size, number of chunks). If Zen sets the dontPersist member to true, it means that the plugin should not save the asset data to disk.

Zen::ZenErrorCode zenQueryAssetDataChunk(Zen::ZenAssetDataChunk* data);retrieves a chunk from Zen. The ZenAssetDataChunk structure is the same as above, you just need to fill in the assetName and chunkIndex members, and Zen will fill chunkSize and chunkData.

So this is the last thing you have to implement: whenever your Zen-hosted plugin needs an asset and can’t find it on the system, it should use zenQueryAssetInformation and zenQueryAssetDataChunk to retrieve the missing asset data from the host.

To be complete, here is the Zen::ZenErrorCode enum:
enum {
ZEN_NO_ERROR,
ZEN_UNKNOWN_ERROR,
ZEN_ASSET_NOT_FOUND_ERROR,
ZEN_OUT_OF_MEMORY_ERROR,
ZEN_METHOD_UNSUPPORTED_ERROR,
} ZenErrorCode;

Testing it

Finally, if you have added the extensions to your plugin, how do you test if it is working fine ? That’s easy:

• Load your plugin in Zen, and import a preset.
• Check the asset chunks are properly imported (they are stored in the Zen folder, under assets/[synth ID])
• Now move away the original sample files from your system, and relaunch Zen
• Select your preset in Zen, and marvel as:
  • Zen instantiates your plugin and sends the preset data.
  • Your plugin doesn’t find the associated sample on the system, and asks the host for help.
  • Zen picks the asset chunks and sends them to the plugin, who uses them to construct the missing asset and flawlessly play the preset.

My last post about the ZenAwareFX API extensions triggered a lot of questions from other developers, and it really seems I need to clarify how things work. Let’s try with the following diagram:

On the right side you can see how Zen works within a VST host: Zen is loaded as a plugin, and can itself load other plugins. These plugins, when supporting the ZenAwareFX extensions, are able to cooperate with Zen, mostly for defining external asset files that the plugin may require.

On the left side, you can see how the ZenDaemon process queries the master Zen database over the internet, to retrieve relevant presets and tags, and update the local Zen database. In most cases (e.g when the host computer is connected to the internet) this ZenDaemon process lives as a separate thread inside the Zen plugin, so there is no need to run it explicitly.

The ZenDB API (currently under definition) will most probably consists in a header file, and a library for you to link with. It will enable plugin developers to query the local Zen database directly – for things like retrieving presets, searching for presets, adding or removing tags…

Basically, this means that a plugin developer who wants to implement a presets browser in his synth will only need to deal with the UI of the browser – tags, presets uploads/downloads/updates will be handled by ZenDB.

The local database is a SQLite file, which means that it doesn’t require to install anything on the end user system.

Another question I get asked a lot is, “if I use ZenDB in my plugin, do I need to install Zen on the end user system ?”. The answer is, “no – but ZenDB will create a Zen-compatible filesystem structure, that Zen will reuse if it is installed at a later time.”

Note: A more recent version of this draft is available here

With Zen becoming more stable (especially with the 1.4 release), it is now time to start on opening it up to other developers. I am going to release 2 Zen API’s:
- the ZenAwareAudioFX API is a small extension to AudioEffectX, designed for cooperation between Zen and its hosted plugins.
- the ZenDB API will provide access to the local Zen database, and will allow developers to run searches to implement their own presets browser/manager.

Let me focus on the first API, which is the one I have mostly been working on lately. The need for this API stemmed from the need to deal with complex presets, with dependencies on data files (typically, samples) managed outside of the plugin. Absynth is a good example, and if you try to load a preset into Absynth without having the required .wav files in the proper folder, you will be greeted by an error message.

The ZenAwareAudioFX class aims to correct this, by enabling the plugin to communicate information about the required external files (that I call “assets”) it needs for the current preset. Everything is based on the existing mechanisms in the VST SDK, namely:

• The host (e.g, Zen) will send requests to the Zen-aware plugin using vendorSpecific calls
• The plugin will query the host using hostVendorSpecific calls

The ZenAwareAudioFX class takes care of all these calls, and leaves you with a few high level methods. These methods are pure virtual, so you simply have to inherit from ZenAwareAudioFX instead of AudioEffectX, and implement these methods in your plugin.

The methods you will need to implement are explained below:

virtual bool hasExternalAssets() const;This tells Zen if your plugin uses external assets. Usually this will return true, as you wouldn’t be here otherwise !

virtual unsigned int zenGetNumberOfAssets() const;This tells how many assets are used by the current preset.

virtual Zen::ZenErrorCode zenGetAssetName(unsigned int idx, char* assetName) const;This returns the name of the nth asset in the current preset. The host (Zen) will provide a char[128] buffer in assetName for you to write in. This name will be used to uniquely identify your asset among others for the same plugin, so if you use numeric ID’s for your assets, you can use itoa or equivalent to fill in the name.

virtual unsigned int zenGetAssetDataSize(unsigned int idx) const;As far as Zen is concerned, assets are opaque blocks of data with a name and a size. Zen will call this to know the required size, so that it can allocate it before calling the next method.

virtual Zen::ZenErrorCode zenGetAssetData(unsigned int idx, void* data) const;This sends the nth asset data to Zen. You just need to write into the provided data area (which has been allocated by Zen according to the size you declared in the previous method), using whatever serialization mechanism you want. Zen will associate this data block with the preset, upload it to the server, and distribute it to the clients.

typedef struct {
const char *assetName;
unsigned int assetSize;
void* data;
} ZenAssetData;
Zen::ZenErrorCode zenQueryAssetData(Zen::ZenAssetData* data);This is used by the plugin to retrieve the asset from Zen. You will actually have to call this twice: first fill in assetName and set data to NULL. The call will return assetSize. Then allocate your memory area according to assetSize, set the data pointer in the ZenAssetData structure, and call this method again – Zen will then fill the area with the asset data.