Multi-Layered Plugins
October 3rd, 2005 by Bernard Lebel - Viewed 4450 times - Popularity: 6% [?]ABSTRACT
In this article I will discuss one way of how can plugins benefit from the XSI 4.0+ plugin architecture. I will look into what I call “multi-layered plugins”, that is, a particular software structure design for XSI plugins. This articles is more about design than technical content.
INTRODUCTION
If you are a TD and do a lot of scripting, chances are that you have already started taking advantage of the XSI 4.0+ plugin architecture, where all kinds of plugins can simply be dropped in the appropriate directory and install themselves by clicking the Update button. These are known as self-installed plugins, and their arrival made the old plugin architecture (toolbars, registered custom commands, spdls, etc) obsolete. In fact, many shader writers would love to get rid of shader spdls and use self-installed plugins instead. As I write these lines, Softimage is working on self-installed compiled custom operators. This is an unstoppable train, you have to catch up with it.
Four months ago I started a new job at Big Bang Digital Studios, in Montreal, as ligthing, rendering and pipeline technical director for the animation crew (Big Bang also has a visual effects crew, which work on other projects and with their own pipeline). My first duty was to “stabilize” the rendering, that is, streamline its workflow and setup its pipeline. This meant establishing norms: naming conventions, directory and file management, passes and partitions, XSI do’s and don’t, as well as creating custom tools, which is what I will discuss here.
So in this four months of work I spent most of my time in a text editor, and creating an enormous library of plugins and Python modules (I do everything in Python). But it quickly became apparent that I could not write single-purpose plugins, in fact I needed to think larger, so the only solution was multi-layered plugins.
THE MULTI-LAYERED PLUGIN
In short, what I call a multi-layered plugin is a plugin that looks like a single script to the user but involves the interaction of many plugins and modules with various degree of specialization. The idea is highly inspired by the XSI scripting API, where some high level commands usually used in scripts call lower level commands. The lower level version can also be called directly, but need more arguments. Let’s look at a typical structure of mine.
The top plugin
The “top” plugin (aka top-layer plugin) lies on a toolbar (it could also be NetView), and serves as a front-end for a set of functionalities hidden from the user. It is clicked by the user to accomplish a specific task. Such tasks include importing characters in a scene, importing animation, exporting cameras, and so on. The top plugin only “catches” the user input (selection of objects, combo boxes, etc), passes it to someone else (the “brain”), and waits for the brain to return.
The brain plugin
The “brain” plugin is somewhat the master plugin. It receives directives from the user via the top-layer plugin in the form of arguments, and takes the appropriate action. It generally means a bit of argument input handling, installing a custom property, setting default parameters in this property, inspecting it, extract parameter values from it. Once this is done, it calls the appropriate function, the “expert”, passes down all needed parameters to the expert, and wait for the expert to return. When it got the return value, it returns to the top plugin.
The brain plugin has a moderate level of specialization. It has to dispatch the information provided to by the user to the right expert. It means the programmer has to always develop this plugin in order to be able to handle every new case that may be needed to be handled.
The interface plugin
While the top-layer plugin handles the “immediate” user input, that is, the selection of objects and printing messages, the crux of the user input actually happens a moment later, and a self-installed custom property is necessary. For the user it seems like clicking the button made this interface pop-up, which is conceptually true, but we know now that this window actually is the result of several plugin calls.
The interface custom property is always installed by the brain plugin. The brain plugin can set some -or all- its parameters, and then inspects it. Since all my custom properties are inspected in modal mode, the brain plugin also has the duty of cleaning the XSI scene before it installs the custom property and after it has finished inspecting it.
There is a reason why the brain sets parameters in the interface before inspecting. I really like highly dynamic interfaces, where the content is as contextual as possible. So the brain sets parameters values before inspection in order to add further customization to the interface, and when the interface is inspected, the user doesn’t have to think much and only a minimal amount of input from him is required. Another reason for this “preset value” is that like brain plugins, interface plugins are often very general, and can fit many different usees.
The expert plugin
The expert plugin acts a bit like the “SI” commands in XSI, where these commands accept extra arguments to perform a task but do little to no error handling. The expert expects the right arguments, otherwise it can’t do its job. Its only purpose is to accomplish its designated task.
Such expert plugins come in many shapes and forms. Here I use the term plugin, but in fact it may either be another self-installed plugin, or a Python module, a class object, or else. It is the job of the brain to call the right expert, and then it is the expert’s job to execute the job the user initially intended for.
Helpers
Throughout this chain of plugins, there is a fair amount of helpers. Helpers are “side” functions, commands, modules, classes, or whatever, that contribute to the execution of the task. Helpers can either be ultra-specialized (like finding all .preset files exported from the Mixer to carry character animation of a shot through the pipeline), to very general (like formatting mapped drives path strings to UNC ones). There is no amount of helpers that can be used in the chain of plugins. Without helpers, many useful functions would have to be written over and over.
EXAMPLE OF A MULTI-LAYERED PLUGIN
Here is an example of a multi-layered plugin. One of my multi-layered toolsets exports XSI scene data to XML text file and can import back this XML data into XSI. Here I will only discuss the part that exports to XML.
All the specialized functions that actually write the XML text are consolidated in a Python module called bb_exportxml (my Python modules are all lowercase and start with “bb_”), wich is made of tens of functions that can call each other at any time, procedurally. These make the “expert” body of the whole plugin. When the brain plugin, named SIExportXMLObjects, is ready to launch the writing task, it will call the appropriate expert function. Even the expert functions are organized in a brain-expert relationship: the brain plugin, most of the time, will call a function named “inputDispatch”, which in turn tests the data to write to XML, and calls the appropriate writing function (these functions always start with a “tag” in their name, to indicate they actually write XML).
The brain plugin can handle any type of XML writing task, as well as the interface plugin (called ExportXMLProperty, all my custom properties end with “Property”). Both have been made very general, and currently hold all the necessary options and arguments to carry any XML writing tasks.
The interface plugin has several built-in functions that are completely transparent to the user. For instance, the user can specify if he is writing shot-specific data or not and used in which project. If this is shot-specific, then a helper plugin is called. This helper, called SIBuildProjectPaths (all my brain and expert plugins start with SI to differentiate them from top-layer plugins), returns a dictionary of paths that follow the norms of the project. Then the interface plugin tests if the path actually exists, if not, it will either inform the user about that or go further and ask him if he wants to create the directories. In short, when the user is exporting shot-specific data, he never has to type a file path. Also, the interface plugin has few callbacks that will update the path field as well as the file name, depending on the shot the user specifies and the type of data he is exporting.
The interface plugin is also a place where “pre-setting” parameters, done by the brain plugin, come into play. For example, if the user wants to export a reference model to XML, the brain sets the checkbox that would make the expert plugin write down the entire tree structure of the model to False (I generally tell the artists to trust me and always leave default options in such interfaces… or come see me). In fact, I’m planning a future version where the layout build will take the value of more parameters into account and will hide parameters I don’t want changed from the user. Another example is if the user is a camera to XML, there is a hidden parameter in the interface that the brain plugin sets and tells the interface to enforce a specific naming convention.
On the front-end of things, all the user sees is a toolbar with a bunch of buttons to perform XML-related tasks. When the user clicks that, then the interface pops-up, so he just needs to specify the various options, and it’s done. Unless he reads the history log (my plugins are fairly verbose to help troubleshooting), the user has no idea of all that what happened.
Note: I may release this set of plugins to the public, as I have clearance from Big Bang to do so. However it will not be until some time, as the design of the toolset is still subject to development and significant changes.
Also, you may be wondering why is bb_exportxml a Python module and not a self-installed command, considering that most of the calls made in the file are done to the same function, inputDispatch(). There are actually two reasons for that. First, having a module instead of a self-installed plugins allows me to set module globals, as opposed to when you deal with a self-installed command, you must pay extra attention about argument passing. Having module globals means the functions can look up the global scope name table and get their global from there, without any further hassle. Second, only a handful of export tools are currently exposed to user. However in the future, I will need to be able to extract any function from the module, bypassing the inputDispatch() function. On the other hand, the self-installed custom command SIBuildProjectPaths may soon be turned into a module, because recently the need to use its return dictionary outside of XSI has arose (we generate discreet combustion compositions with a command line script).
PROS
- Extremely flexible, customizable and adaptable
- Structure completely modular
- Maximum code reuse
- Takes advantage of the self-installed plugin architecture
- Consolidation of plugins by family
CONS
- Highly complex architecture and chain of dependencies
- Difficult to make design changes, especially once the plugins are being used in the pipeline
- Can have a measurable performance cost (especially when used through network workgroups)
CONCLUSION
Despite the cons I have mentioned, I think the pros outweigh these by far. In fact I’m a big fan of such structures, as I think the flexibility and modularity is the greatest strength offered by this design. I for sure will keep developing such plugins. However, since the design has disadvantages, I think one must rely on it only when needed. I was advised some time ago, when I was learning Python, to use classes only when necessary. I think this was a wise advice, and I think it also applies to pretty much everything related to programming. If the benefits of a design are not needed, why running the risk of letting the sh*t hit the fan because of its disadvantages?
Popularity: 6% [?]
