In Softimage 2010, a new lip sync feature was added to Face Robot (here’s the doc).  In 2011 SP1, there are three new commands to get the data out into your own tools.  One of them doesn’t even require the use of Face Robot and calls the speech recognition library directly.

The commands return VBScript-compatible multi-dimensional arrays.    This is transparent in VBScript and Python, but it requires a bit of magic in JScript, so check the example in the first command if that’s what you’re using.

Getting Phonemes without Face Robot

SIExtractPhonemes

Description

Scripting Syntax

oReturn = SIExtractPhonemes( Language, FileName, GuideText );

Return Value

Parameters

Examples

//returns a VBArray object, which require using the .getItem method in Jscript because they are not
//native objects there.
//However, they work transparently like multi-dimentionnal arrays in VBscript and Python
var res = SIExtractPhonemes ("en", "s:\\here_be_dragons.wav", "Here be dragons, the map says");
arraysize = res.ubound( 1 );
logmessage( arraysize + "phonemes found" );
for (  i=0; i < arraysize; i++)
{
logmessage( "IPA#" + res.getItem(i,0)
         + "(" + res.getItem(i,3) + ") "
         + "Start Time(sec) " + res.getItem(i,1)
         + "End Time(sec) " + res.getItem(i,2) );
}

Getting Lip Sync Stuff Out of Face Robot

This allows the artists to tweak the phonemes with the Face Robot UI, but used the data outside of Face Robot, perhaps in a game engine that does phoneme blending already.

The second command even allows you to get the result of the blend algorithm of Face Robot, so you can make a simplified playback environment.

GetPhonemeFromSpeechClip

Description

Scripting Syntax

oReturn = GetPhonemeFromSpeechClip( SpeechClip );

Return Value

Parameters

Examples

phonemes = GetPhonemesFromSpeechClip ("Face.Mixer.Phoneme_Speech.SpeechAction_Clip")
LogMessage "phoneme data"
for i = 0 to UBound( phonemes )
	LogMessage "IPA # " & phonemes(i, 0) & "(" & phonemes(i, 3) & ")" & " Start (sec)" & phonemes(i, 1) & " End (sec)" & phonemes(i, 2)
next

GetPhonemeBlendFromSpeechClip

Description

Scripting Syntax

oReturn = GetPhonemeBlendFromSpeechClip( SpeechClip, [Channel], [Time] );

Return Value

Parameters

Examples

phonemes = GetPhonemeBlendFromSpeechClip ("Face.Mixer.Phoneme_Speech.SpeechAction_Clip")
LogMessage "phoneme blend"
for i = 0 to UBound( phonemes )
	LogMessage "IPA # " & phonemes(i, 0) & "(" & phonemes(i, 1) & ")" & " Weight " & phonemes(i, 2 )
next

I put some effort into commenting the script accordingly, so that it may be used for learning purposes.

cheers!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148

// query the geometry data
var obj = selection(0);
var geo = obj.Activeprimitive.Geometry;
var polygons = geo.Polygons;
 
// create an array to track which polygon has been visited
var polygonVisited = [];
var polygonsLeft = polygons.Count;
 
// fill up the array to say: "no polygon has been visited yet"
for(var i=0;i<polygons .count;i++)
{
	polygonVisited[i] = false;
}
 
// create some helper variables
var clusterIndices = [];
var currentRow = [];
 
// log the start of the process
logmessage("Starting with "+polygonsLeft+" polygons...");
 
// loop until there are no polygons left
// or there are still items in the cluster array
while(polygonsLeft > 0 || clusterIndices.length > 0)
{
	// if there are no items in the next row...
	if(currentRow.length == 0)
	{
		// if there are items in the cluster index array
		if(clusterIndices.length > 0)
		{
			// log the creation of the cluster
			logmessage("Creating cluster with "+clusterIndices.length+" elements...");
 
			// create a new mesh containing all of these polygons
			// first, create an array containing all pointindices
			// and two additional point hash arrays
			// of which one is used to determine if a point is already used
			// and the second one is used to map the original index to the new one
			var pointIndices = [];
			var pointUsed = [];
			var pointNewIndex = [];
 
			for(var i=0;i<clusterIndices.length;i++)
			{
				var poly = polygons.item(clusterIndices[i]);
				var pointIndexArray = poly.points.indexArray.toArray(); //(VBS to JScript conversion)
 
				// loop through all points and check if it they are used
				for(var j=0;j<pointIndexArray.length;j++)
				{
					var index = pointIndexArray[j];
					if(!pointUsed[index])
					{
						pointNewIndex[index] = pointIndices.length;
						pointIndices.push(index);
						pointUsed[index] = true;	
					}
				}
			}
 
			// now, create the array of the positions of all points of this island
			var islandPos = [];
			for(var i=0;i<pointIndices.length;i++)
			{
				var point = geo.points.item(pointIndices[i]);
				islandPos.push(point.position.x);
				islandPos.push(point.position.y);
				islandPos.push(point.position.z);
			}
 
			// now we need to create the description of the polygons
			var islandPoly = [];
			// for that we loop over the polygons of the island again
			// and record their pointindices, well, actually the remapped
			// indices (given by the pointNewIndex hash array)
			for(var i=0;i<clusterIndices.length;i++)
			{
				var poly = polygons.item(clusterIndices[i]);
				var pointIndexArray = poly.points.indexArray.toArray(); //(VBS to JScript conversion)
 
				// first add the number of points of this polygon 
				// to this polygon's description
				islandPoly.push(pointIndexArray.length);
 
				// now add the remapped index for each point to the description
				for(var j=0;j<pointIndexArray.length;j++)
				{
					var index = pointIndexArray[j];
					islandPoly.push(pointNewIndex[index]);
				}
			}
 
			obj.Parent.AddPolygonMesh(islandPos,islandPoly,obj.name+"_island");
			clusterIndices = [];
 
			// if there are no polyons left we can stop now
			if(polygonsLeft == 0)
				break;
		}
 
		// loop over the polygons and try to find 
		// one which hasn't been visited yet
		for(var i=0;i<polygons.Count;i++)
		{
			if(polygonVisited[i] == false)
			{
				currentRow.push(i);
				clusterIndices.push(i);
				polygonVisited[i] = true;
				polygonsLeft--;
				break;
			}
		}
	}
 
	// define the nextrow after the current row
	var nextRow = [];
 
	// loop over all items in the current row
	for(var i=0;i<currentRow.length;i++)
	{
		var polygon = polygons.item(currentRow[i]);
		var neighbors = polygon.NeighborPolygons();
 
		// loop through all of the neighbors and add them to the nextRow
		for(var j=0;j<neighbors.count;j++)
		{
			// check if the neighbor has been
			// visited yet
			var neighbor = neighbors.item(j).index;
			if(polygonVisited[neighbor]==false)
			{
				// add the neighbor to the next row
				nextRow.push(neighbor);
				clusterIndices.push(neighbor);
				polygonVisited[neighbor] = true;
				polygonsLeft--;
				break;
			}
		}
	}
 
	currentRow = nextRow;
}
 
selection.SetAsText(obj.name+"_island*");

Let’s get a curve

We start by taking a poly strip, with a 100×1 subdivisions. This will be our “drawing” table. We then make a cluster of the whole set of points, and give it a weightmap. The range for the weights is set as wide as possible (-100, 100) to allow for both large positive and negative values. We then apply to the grid a custom property with a profile function curve, which you can then shape as you wish.

Here is a script that will do the steps mentioned above.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

function CreateGrid()
{
    //create a poly strand, 100x1
    var grid = CreatePrim("Grid", "MeshSurface", null, null);
    SetValue("grid.grid.ulength", 10, null);
    SetValue("grid.grid.vlength", 1, null);
    SetValue("grid.polymsh.geom.subdivu", 100, null);
    SetValue("grid.polymsh.geom.subdivv", 1, null);
 
    //create the cluster
    SelectGeometryComponents("grid.pnt[*]");
    CreateCluster(null);
    //apply a weightmap
    CreateWeightMap(null, null, "Weight_Map", null, null);
    SetValue("grid.polymsh.cls.Point.Weight_Map.red", 0, null);
    SetValue("grid.polymsh.cls.Point.Weight_Map.blue", 0, null);
    SetValue("grid.polymsh.cls.Point.Weight_Map.green", 0, null);
 
    //allow for large min and max weights
    SetValue("grid.polymsh.cls.Point.Weight_Map.weightmapop.wmin", -100, null);
    SetValue("grid.polymsh.cls.Point.Weight_Map.weightmapop.wmax", 100, null);
 
    return grid;
}
 
 
function ApplyProfile(grid)
{
    var oCPSet = grid.AddProperty("CustomProperty", false, "ProfilePSet");
    var oFCParam = oCPSet.AddFCurveParameter("ProfileFC");
 
    var oFCurve = oFCParam.Value;
 
    oFCurve.SetKeys(new Array(0, 0, 0.25, 1, 0.5, -0.5, 0.75, 0.8, 1, 0));
}
 
var grid = CreateGrid();
ApplyProfile(grid);

Sampling the curve

This is done by the following script.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

function SampleFunction(pSet)
{
    var par = pSet.NestedObjects(0);
    var fC = par.value;
 
    var pArray = [0,0];
    var vArray = [0,0];
 
    //fcurve min and max range
    var rangeMin = 0;
    var rangeMax = 1;
 
    var range = rangeMax - rangeMin;
 
    var x, y;
 
    //number of samples, equal to the grid number of points along x
    var nbSamples = 100;
 
    for (i=0; i< =nbSamples; i++)
    {
        //i-th column of points
        pArray[0] = i*2;
        pArray[1] = i*2 + 1;
 
        x = rangeMin + range * (i/nbSamples);
        y = fC.Eval(x); //value of the function at x
 
        vArray[0] = vArray[1] = y; //values for the weight map
 
        //set the weights for the i-th column of points
        PaintWeights("grid.polymsh.cls.Point.Weight_Map", pArray, vArray, 1, 0);
    }
}
 
//select the ProfilePSet first
var profilePSet = selection(0);
SampleFunction(profilePSet);

You just have to take care of setting the current range min and max correctly and then run it with the ProfilePSet. The script will simply sample the curve with the same rate as the grid subdivisions along x. The grid has 101 column of points along the x. We take the corresponding function values and stick them on the weightmap. Note that we’re using 100 samples, but we could have used much more for both the grid and the sampling rate.

Plotting the weightmap

We apply a color sampler lightmap shader. The input color to be sampled comes from a scalar lookup of the weightmap.

The writable map must be a hdr one (float .ct is my choice), in order to allow for large positive and negative values to be stored. Only the X resolution matters, since we are basically plotting a mono dimensional texture. The map is applied with a xz projection over the grid. You now just need to fire a tiny render region to have the lighmap written.

Reusing the map

As said, the map is one dimensional, so only the u matters when you look it up. So, if you pick a share vector node and connect it to an image lookup, the x coordinate of the vector acts as the input you are remapping by the original function curve. In this example, the lookup feeds the displacement, being the value to be remapped coming from a standard uv projection over the new grid.

What is important is to remember the range the function was originally sampled over, that must match the expected range of the value (the u coordinate in this example) you are going to remap.

Sampling a scripting function

As mentioned, you can also use this approach to paint any math function provided by your scripting function. Instead of using a function curve, you can simply loop and evaluate the desired function.

For instance, this is how you sample and weight paint a sin function in the -pi +pi range:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

function SampleSin()
{
    var pArray = [0,0];
    var vArray = [0,0];
 
    var rangeMin = -3.14159;
    var rangeMax = 3.14159;
 
    var range = rangeMax - rangeMin;
 
    var x, y;
 
    var nbSamples = 100;
 
    for (i=0; i< =nbSamples; i++)
    {
        pArray[0] = i*2;
        pArray[1] = i*2 + 1;
 
        x = rangeMin + range * (i/nbSamples);
        y = Math.sin(x); //value of the function at x
 
        vArray[0] = vArray[1] = y; //values for the weight map
 
        PaintWeights("grid.polymsh.cls.Point.Weight_Map", pArray, vArray, 1, 0);
    }
}

Since the Histogram display window is just a COM application launched by the equivalent mental ray shader, there’s nothing stopping you using the Histogram display tool for other uses. Indeed, I included in the original download example source scripts to demonstrate how to drive the COM interface.

Vince Fortin has asked me about whether it’s possible to get Histogram to show distributions for particle systems. So at his request (thanks for reminding me Vince!) I’m posting some example code to show you how to do it.

First though, here’s a quick screen shot of the sort of information it can show:

Here I’m using Histogram to show the distribution of the X, Y, and Z positions of the particles. I’ve also got it to display the particle age too. Note that the number of samples (1853) shown at the bottom, actually corresponds to the number of particles.

The JScript code below features a reusable object called HistogramData. This object has a member function called ProcessSample() which will take a sample value and take care of the histogram construction for you. You then push the curHist member to the Histogram COM application for display.

The first function in the code is a script callback for a particle event. To make it work, you just need to add a particle event to a PType and create an OnEveryFrame event with the trigger value set to 0 so that the script gets called every frame. You can then control the execution of the Histogram display inside the script by changing the trigger frame number.

You can download and find out more about Histogram by following this link:

www.andynicholas.com -> Histogram

Please feel free to modify this code for your own usage.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151

var HISTOGRAM_RES = 256;
var INV_HISTOGRAM_RES = 0.00390625;
var HISTOGRAM_ARRSIZE = 257;
 
function OnEveryFrame(inParticleCloud, inTriggerParticleIndices, inSimFrame)
{
//Frame 99 is when the distribution will be viewed, and
//you can change it to whatever you want.
 
//Note that if you persist the histogram data objects, there's no
//reason you can't gather distributions across multiple frames.
 
//To make things a little easier to adjust, you may also want to
//create a custom property in the scene to tie this frame value to.
 
if(inSimFrame==99)
{
//Initialise the data
var histx = new HistogramData();
var histy = new HistogramData();
var histz = new HistogramData();
var histage = new HistogramData();
 
//This is where we store the data whose distribution we want to look at.
//In this case, we're observing position and age.
 
var particles = inParticleCloud.particles;
var num_particles = particles.count;
for(var i=0;i<num_particles;++i)
{
var particle = particles(i);
histx.ProcessSample(particle.position.x);
histy.ProcessSample(particle.position.y);
histz.ProcessSample(particle.position.z);
histage.ProcessSample(particle.age);
}
 
//Display the data
var histWindow = new ActiveXObject("HistogramDisplay.Application");
histWindow.SetHistogram(histx.curHist,"Pos.X", 0, histx.min, histx.max);
histWindow.SetHistogram(histy.curHist,"Pos.Y", 1, histy.min, histy.max);
histWindow.SetHistogram(histz.curHist,"Pos.Z", 2, histz.min, histz.max);
histWindow.SetHistogram(histage.curHist,"Age", 3, histage.min, histage.max);
 
histWindow.SetHistogramColor(0, 80, 0, 0);
histWindow.SetHistogramColor(1, 0, 80, 0);
histWindow.SetHistogramColor(2, 0, 0, 80);
histWindow.SetHistogramColor(3, 0, 0, 0);
 
histWindow.SetNumToDisplay(4);
histWindow.UpdateWindow();
}
}
 
//Data object representing histogram data
//and contains member function called ProcessSample()
 
function HistogramData()
{
this.min=0;
this.max=0;
this.samples=0;
 
this.curHist = new Array(HISTOGRAM_ARRSIZE);
this.otherHist = new Array(HISTOGRAM_ARRSIZE);
 
for(var i=0;i<HISTOGRAM_ARRSIZE;++i)
{
this.curHist[i]=0;
this.otherHist[i]=0;
}
 
this.ProcessSample = function(sample)
{
//This function does all the hard work
 
//It is a direct port from C++ of the functionality
//of the original Histogram rendertree shader
 
var oldMin = this.min;
var oldMax = this.max;
var oldBucketSize = (oldMax-oldMin)*INV_HISTOGRAM_RES;
 
var scaleChanged=false;
 
if(this.samples==0)
{
this.min = sample;
this.max = sample;
this.samples++;
return;
}
 
if(sample<this.min)
{
this.min = sample;
scaleChanged=true;
}
else if(sample>this.max)
{
this.max = sample;
scaleChanged=true;
}
 
if(this.samples==1)
{
this.curHist[0]++;
this.curHist[HISTOGRAM_RES]++;
this.samples++;
return;
}
 
var bucketSize = (this.max-this.min)*INV_HISTOGRAM_RES;
 
var rangeRatio = HISTOGRAM_RES/(this.max-this.min);
if((this.max-this.min)==0) rangeRatio=0;
 
if(scaleChanged)
{
//Swap histograms
var tempHist;
tempHist = this.curHist;
this.curHist = this.otherHist;
this.otherHist = tempHist;
 
//Copy and rescale the histogram according to the new sample and resample
for(var i=0;i<HISTOGRAM_ARRSIZE;i++)
{
//Calculate sample value
var value = oldMin + oldBucketSize*i;
 
//Calculate new bucket
var newBucket = Math.floor((value - this.min)*rangeRatio);
 
if(newBucket<0) newBucket=0;
if(newBucket>=HISTOGRAM_ARRSIZE) newBucket=HISTOGRAM_ARRSIZE-1;
 
this.curHist[newBucket]+=this.otherHist[i];
this.otherHist[i]=0;
}
}
 
//Add sample
var inputBucket = Math.floor((sample - this.min)*rangeRatio);
if(inputBucket<0) inputBucket=0;
if(inputBucket>=HISTOGRAM_ARRSIZE) inputBucket=HISTOGRAM_ARRSIZE-1;
this.curHist[inputBucket]++;
 
this.samples++;
}
}

Fortunately XSI gives us the possibility to customize a custom property’s layout with groups, tabs, by changing the parameter’s aspect, and replicating them as many times as we wish.

Unfortunately these layout informations are not persisted with the scene which means they are forgotten from one XSI session to another.

The idea

To bypass this problem I wrote a self installing custom property in JScript. The property has a string parameter where I can write my layout code and the property can read it and change itself. Because this code is actually a parameter’s value it is saved in the scene which means we are not going to loose the layout.

A look at the code

The first thing we need to create is a plugin that registers a new property. We can use the wizard in XSI to do this and call the new property whatever we want. I called it GenericProperty because, well, it’s generic.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

function XSILoadPlugin(in_reg)
{
	in_reg.Author = "Daniele Niero";
	in_reg.Name = "GenericProperty_Plugin";
	in_reg.Major = 1;
	in_reg.Minor = 0;
	in_reg.RegisterProperty("GenericProperty");
	return true;
}
 
function GenericProperty_Define(io_Context)
{
	var oCustomProperty;
	oCustomProperty = io_Context.Source;
	oCustomProperty.AddParameter2("PropertyCode",siString,"",null,null,null,null,0,siPersistable);
	oCustomProperty.AddParameter2("PropertyPPGTab", siBool, true);
}

The trick is performed in the OnInit function

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

function GenericProperty_OnInit()
{
	var oLayout = PPG.PPGLayout;
	var startCode = "function ChangeProperty(){\r\n" + "oLayout.Clear();\r\n";
	var userCode = PPG.PropertyCode.Value + "\r\n";
 
	if (PPG.PropertyPPGTab.Value == true)
	{
		var endCode = "oLayout.AddTab(\"PPG\");\r\n" + "oLayout.AddItem(\"PropertyPPGTab\");\r\n" + "oLayout.AddString(\"PropertyCode\",\"PropertyCode\", true, 500);\r\n" + "}";
	}
	else if (PPG.PropertyPPGTab.Value == false)
	{
		var endCode = "}";
	}
 
	var code = startCode + userCode + endCode;
	eval(code);
	ChangeProperty();
	PPG.Refresh();
}

There are three important variables in this code startCode, endCode and especially userCode. They are strings that define the layout code.

startCode is a constant and it will never change. As you can see it defines a function called ChangeProperty(), all the rest of the code is part of this function.

userCode defines the code that creates a tab called PropertyPPGTab with a parameter called PropertyCode and then closes the ChangeProperty function. The tab is defined only if PropertyPPGTab is true, allowing us to hide PropertyCode Errors are thus prevented as the custom property is left perfectly clean and animators aren’t scared with strange code they normally don’t want to see.

userCode is the variable we are interested in. It is filled with the value of the PropertyCode parameter and in this parameter we can write our layout code directly from the property in XSI.

At the end all these three variables are connected together in a single one called code and evaluated with the eval function. If code doesn’t contain any syntax errors eval will evaluate it successfully and we can then call the ChangeProperty() function to have it create our layout.

Final touch, PPG.Refresh() ensures a correct layout refresh.

How use it

Very simple, just load the plugin in XSI and apply this property on an object, open it and create some custom parameters or proxy parameters as usual and start to write your layout.

For example imagine you have two parameters (custom or proxy doesn’t make any difference) named Param1 and Param2.

We can write a layout like this:

1
2
3
4
5
6
7
8
9
10

oLayout.AddGroup("First Parameter");
	oLayout.AddItem("Param1");
oLayout.EndGroup();
oLayout.AddGroup("Second Parameter");
	oLayout.AddItem("Param2");
oLayout.EndGroup();
 
oLayout.AddTab("A Tab");
	oLayout.AddItem("Param1");
	oLayout.AddItem("Param2");

Refresh or just reopen the property and you should be able to see your layout.

Limitation

I have been using this solution from almost two years now and it has worked really well so far. Recently I wanted to improve it a little bit and I tried to implement the possibility to write a logic session as well.

Unfortunately it seemed impossible to make the logic as dynamic as the layout. I had problems flushing and regenerating the Logic even if using oLayout.Delete() instead of just Clear().

Okay so here’s how it works:

1. Create your geometry and rig
2. Put rig and geometry into two different models
3. Create the connection (envelope, constraints etc)
4. Export FIRST the geometry, and SECOND the rig.
5. Go to a new scene and import FIRST the rig and SECOND the geometry as referenced models.
6. To ensure the models are loaded in the right order in the future, also name them alphabetically in that order. (in my example I am using numbers to set that order)
7. Put both ref models into a new one and export it as the “fullmodel”.

You end up with three files. The first one includes just the geometry + envelope operator and weightmaps, the second one includes just the rig, and the third one, the “fullmodel”, includes just two file pointers to the previous two.

If you want to go ahead and make changes to the rig, you can externally, while just loading the rig up and modify it. If you are about to change the geometry, and you don’t want to loose the envelope, just open up the fullmodel, localize the submodels and do your edits, afterwards convert the geometry back to referenced, and reload the “fullmodel”.

Below you can see a working example in the explorer, and a repro script to generate this setup. All steps (also changes on the geometry after the connection has been established) are part of that script.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

var project = ActiveProject.path;
NewScene(false,false);
 
// create objects
CreatePrim("Cylinder", "MeshSurface");
Create2DSkeleton(0, 3, 0, 0, 0, 0.5);
AppendBone("eff", 0, -3, 0);
 
// create models
CreateModel("root", "_0rig");
CreateModel("cylinder", "_1geo");
 
// reset the transform of the models so
// they are in the center of the world
SetUserPref("SI3D_NODETRANSFORM_CHILD_COMPENSATE", 1);
ResetTransform("_0rig", siObj, siSRT, siXYZ);
SetUserPref("SI3D_NODETRANSFORM_CHILD_COMPENSATE", 0);
 
// create the envelope (connect the rig)
ApplyFlexEnv(
"_1geo.cylinder;"+
"_0rig.bone,_0rig.bone1"
);
 
// export to files
ExportModel("_1geo", project+"\\Models\\geo.emdl");
ExportModel("_0rig", project+"\\Models\\rig.emdl");
 
// new scene
NewScene(false,false);
 
// import both as referenced
SICreateRefModel(project+"\\Models\\rig.emdl", "_0rig");
SICreateRefModel(project+"\\Models\\geo.emdl", "_1geo");
 
// create a new model including both
CreateModel("_0rig,_1geo", "fullmodel");
 
// export this model again
ExportModel("fullmodel", project+"\\Models\\fullmodel.emdl");
 
// new scene
NewScene(false,false);
 
// import the fully rigged, separated model as referenced
SICreateRefModel(project+"\\Models\\fullmodel.emdl", "fullmodel");
 
// change the geometry while maintaining the connection
MakeModelLocal("_0rig", null);
MakeModelLocal("_1geo", null);
Translate("_1geo.cylinder.edge[67]", 0,0, -1.5);
ConvertToRefModel("_1geo", project + "\\Models\\geo.emdl");
 
// reload the full model
UpdateReferencedModel("fullmodel");

Here’s a link to the script file so you can download it and try it out.

I hope this helps!
Helge

I was required to create a bubble stream behind an underwater torpedo. Do torpedoes really have a stream of bubbles behind them in real life? In movies they do, so the question doesn’t really matter. I wanted to have many small bubbles and a few big bubbles within the same PType. I wanted to have the big bubbles rise faster than the small ones. I also wanted the bubbles to wobble as they were rising, and the wobble to be function of the size and the speed. Finally, I wanted the sprite sequence to begin at a random frame and continue normally as the particle aged. All this is not possible with the actual particle system in XSI 5.11, although they are common requirements in a simulation scene.

The Distribution problem.
In XSI, you can only choose from uniform and Gaussian distribution. This means you can only have a middle value, and an equal probability on each side. For what we need here, we will create a scripted event with the trigger on “Particle Age” at zero. We will name the event “BirthControl“. In the script tab, we set the language to JScript. It’s a shame we can’t use Python, since there are many cool random distributions in the standard library. The ScriptContext will be set to “Per Trigger Particle”. An exponent function will give a random distribution with a high probability at one end and a progressively lower probability towards the other end.

Size = (random number)5 * 0.025 + 0.005
The “* 0.025″ sets the difference in size between the biggest and the smallest bubbles.
The “+ 0.005″ sets the size of the smallest bubble size at 0.005 units. The biggest bubbles will be 0.03 units big.

In JScript, it’s written like this:

1

inParticle.Size = Math.pow(Math.random(), 5) * 0.025 + 0.005;

The object inParticle is given to the script by XSI and contains the current particle that triggered the event.

The Speed from Size problem.
The size of the particles can be determined randomly. The effect of gravity can also be variable among the particles. But we can’t set a relation between the two with the tools in the UI. In fact, in scripting we don’t even have access to the effect of forces per particle. We will need to limit the length of the velocity vector according to the size of each particle. We must create a new scripted event called “PerFrameControl” with the trigger at “Every N Frame” and the value at 1. The ScriptContext will be “Per Particle”.

If a particle’s speed exceeds the product of it’s size and a coefficient, bring it back to that product.

1
2
3
4
5
6
7

speed = inParticle.Velocity;
size = inParticle.Size;
if (speed.Length() > size * 10) {  // if speed is bigger than 10 times the size...
    speed.NormalizeInPlace();      // scale the speed to be 1 and
    speed.ScaleInPlace(size * 10); // scale speed to be 10 times the size.
}
inParticle.Velocity = speed;

This has the advantage that a particle will have no speed at its birth, and slowly accelerate from the effect of gravity. It will continue to accelerate until it reaches the speed it is allowed. Then it will keep this speed forever unless it has to decelerate, which shouldn’t happen in our scene.

The Wobble Problem
There is already some perlin noise in the PType to make the bubbles rise in a natural fashion and we don’t want to lose that. We can’t add a small brownian noise to simulate the wobble and even if we could, we couldn’t link it to the size and speed of the particles. Without going into complicated wobble behavior, we will simply randomize the particle position in X and Z over time.

Generate a vector that has a random value for the X and Z axes, and multiply that value by the size and the speed. Add that vector the the particle’s position.
In the PerFrameControl event, we add:

1
2
3
4

pos = inParticle.Position;
wobble = XSIMath.CreateVector3((Math.random() - 0.5) * size * speed.Length(), 0, (Math.random() - 0.5) * size * speed.Length());
pos.AddInplace(wobble);
inParticle.Position = pos;

The big particles will wobble more than the small ones, and all particles will wobble progressively more as they gain speed.

The Sprite Sequence Problem.
We have a sequence of images of a bubble rising to the surface to apply on the sprite shader of the particles. It is 100 frames long. There are many choices for the behavior of the sprite sequence on the particles, but an important one is missing. We want each particle to choose a random frame in the sequence at its birth, and take the next frame of that sequence at each frame it ages. For this, we need to create a new User Parameter on the PType. We name it “BeginSpriteSeq” and set its type to Integer.

We set this parameter to a random value between 0 and 100, the length of the sequence, on each particle. In the BirthControl event, we add the following line:

1

inParticle.Attributes("BeginSpriteSeq").Value = Math.random() * 100;

Then, we need to tell the particle to choose the next frames over time. In the PerFrameControl event, we add the line:

1

inParticle.SpriteIndex = inParticle.Attributes("BeginSpriteSeq").Value + inParticle.Age;

I know there are ways to fake this in the render tree but the pseudo-random value is not very easy to obtain and it makes a recipe that is more complicated to update from one situation to another. However, contrary to the scripted event solution, it does not increase the simulation time.

The Result.
After all the work, we end up with only two events that solve all the problems encountered with a reasonable simulation time.

BirthControl event
trigger: Particle Age = 0
ScriptContext: Per Trigger Particle

1
2

inParticle.Size = Math.pow( Math.random(), 5 ) * 0.025 + 0.005;
inParticle.Attributes("BeginSpriteSeq").Value = Math.random() * 100;

PerFrameControl event
trigger: Every N Frame = 1
ScriptContext: Per Particle

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

//speed control
speed = inParticle.Velocity;
size = inParticle.Size;
if (speed.Length() > size * 10) {  // if speed is bigger than 10 times the size...
    speed.NormalizeInPlace();      // scale the speed to be 1 and
    speed.ScaleInPlace(size * 10); // scale speed to be 10 times the size.
}
inParticle.Velocity = speed;
 
//wobble
pos = inParticle.Position;
wobble = XSIMath.CreateVector3((Math.random() - 0.5) * size * speed.Length(), 0, (Math.random() - 0.5) * size * speed.Length());
pos.AddInplace(wobble);
inParticle.Position = pos;
 
//sprite sequence
inParticle.SpriteIndex = inParticle.Attributes("BeginSpriteSeq").Value + inParticle.Age;

Here is a Quicktime movie of the result.
Here is a close-up from the HD version.

In the final version of this recipe, I had to change the script context of the events because I wanted to add a custom PPG on the cloud with some parameters to control the various effects. This required a Dictionary.GetObject() to get the values of those parameters in each event and I didn’t want to evaluate it for each particle. I set the script contexts to Per Cloud, and did some conditional expressions to replace the triggers.

I’m glad we can solve these problems since they occur surprisingly often in production. However, I wish the solutions were more accessible from the UI. Not only would it make it easier for an artist to use them, but also the simulation times might be decreased since it would be part of the internal solver. I hope the good people at Softimage consider those problems in the new particle system they are preparing. The ability to customize the distribution of random values on a parameter is essential to a good simulation package. So is the ability to link random values from one parameter to another.

Lets start. One easy way to approach this problem is to create a mesh including all objects we want to envelope, and then simply set the envelope weights of all points of a single instance to the exact same weight, aka:

In this image we have a mesh with 6 cubes (all merged together) enveloped to a two-bone chain.
By default the weight is blended in the middle part,

which results in a deform for the middle two cubes:

by using the weight editor, I adjusted the weight of all cubes so they only target one bone:

which results in movement, but not deformation for each one:

So this is all working, why bother dealing with a system to deal with these things? Well, as long as we are dealing with six cubes it is all cheesy and simple, but as soon as we start dealing with something like feathers on a bird, we can’t just go in and envelope each feather with the weight editor, well, we could, but it would just take forever.

What I used to do in those cases is to build an instancer operator, which would take a mesh containing “where-and-how-to-instance” information, and a single mesh which is supposed to be instanced. First, lets clarify the “where…to-instance” part:

When instancing an object, all you need is a position, orientation and scaling, which can be stored as three points really, but in this example I am going to use 4 points:

1. Pos: The position of the object to be instanced
2. Cns: The Z-Direction of the object (seen from pos)
3. Upv: The Y-Direction of the object (seen from pos)
4. Bnd: An additional point to be used for deformation (later)

So basically we could draw a quadrangular polygon instead of four points, like below, where we are instancing ten grids:

Now: Instead of enveloping the objects we want to be instanced, we envelope the mesh containing the quads. This is going to be called our “Instancing-Cloud”. It contains all of the information we need to instance our objects. Moreover, it is easy to envelope, as we know exactly which points are to be enveloped where. Every first out of four points is the position, every second out of four points is the z-direction etc… All we need to build is a couple of scripts:

1. One which adds a quadrangle to a mesh based on a transform of a null.
2. One which creates a new mesh and instances an object onto an Instancing-Cloud.
3. One which generates clusters for the instance-cloud for easier enveloping.

A finished setup could look something like this: We see the quads representing the instances:

When deformed: The quads deform, but the instances will not:

Including the instances it should look like this:

When deformed:

As we have point clusters for each type of point in the instance-cloud (pos, cns, upv and bnd) it is quite simple to rig the cns for example to a bunch of rotational controllers:

And when we rotate them, only the cns points are deformed, therefore the instances rotate as well:

Additionally we can deform the cloud overall:

So now lets discuss the last part of the story: What’s the bnd position for?
The bnd position is used for deformation of each instance. As this is quite limited, all I am going to do is move the instances points using a sinus function, to get somewhat of twisting and bending going on. This is up to your imagination, as I said, this setup has been used for feathers, that’s why the bnd position is used for bending (and named like that as well: bnd = bending).

In the next image I rigged the bnd position to the rotational controller earlier used for cns, to show what happens:

So now when I rotate them:

Or along another axis:

As always – I’d like to share the progress so far, but I have to warn you, it is not at all documented:

Put the file into one of the plugins folders, and run the following script to get some feather setup going:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

NewScene(false, false);
var cloud = mt_createFeatherCloud();
 
var grid = CreatePrim("Grid", "MeshSurface", null, null);
SetValue(grid+".grid.ulength", 0.4);
SetValue(grid+".grid.vlength", 3);
SetValue(grid+".polymsh.geom.subdivu", 1);
Translate(grid+".pnt[*]", 1.4, 0, 0, siRelative, siLocal, siObj);
FreezeObj(grid);
 
var mesh = mt_createFeatherMesh(cloud, grid);
var feathers = new ActiveXObject("XSI.Collection");
for(var i=0; i<10; i++)
{
feathers.Add(mt_createFeatherNull());
feathers(i).kinematics.local.posx.value = i;
}
 
mt_addFeathersToCloud(cloud,feathers);
mt_createCloudClusters(cloud);
 
DeselectAll();

Let me know how it goes,

enjoy!

Initially, figuring out how this could work seems like a pretty tricky problem. The bone chain needs to stretch intuitively to give the impression that it’s stuck to the IK effector. But we also need it to flatten gradually to avoid the snap effect. Unfortunately, the only thing we have available to us to control these two effects, is the ability to change the length of the bones, since the end position of the chain must always be located at the effector.

So we need to scale the bones to allow them to stretch, but at a specific rate to make sure we avoid the snap. Like this:

The first thing to do is to look at how we solved the problem for the non-stretchy bone chain. When we look at what happens when we move the Soft Effector past the point where it starts to leave the original IK effector behind, all we want to do for the stretchy bone chain system is make sure that the bone chain gets longer so that the original IK effector is at the same position as the Soft Effector. But we need to do this while also keeping the bones angles the same, this avoids the snap effect since the bone angles would change at exactly the same rate as in the non-stretchy bone chain system.

A diagram should make things a bit clearer. Basically, given a non-stretchy chain, we want to match the position of the two effectors by scaling the chain length, but without changing the bone angles, like this:

Okay, so that doesn’t seem too hard. We just need to figure out how much to move the IK Effector to match the position of the soft effector.

The only way we can do this is by changing the length of the bone chain. Again, this initially looks like quite a tricky problem, until you realize that scaling the length of each bone by some value X, also scales the distance of the original IK effector from the IK root by a factor of X as well. A simple test proves it (and it can be easily shown with a little math too):

This diagram shows a bone chain with bones of different lengths, before and after I scaled each individual bone length by a factor of 2. It can be seen that the IK effector has traveled directly away from the IK root and has doubled it’s distance from it. Not only that, but the bone angles have remained unchanged. This is exactly what we need!

Now, all we need to do is to figure out the amount we need to scale each bone length to make the original IK effector match the position of the soft effector. That scale factor is just the ratio of the distances of the two effectors:

Scale Factor = EvalDistance(IK root, Soft Effector) / EvalDistance(IK root, original IK effector)

Where EvalDistance() is just a function that measure the distance between the two objects.

Finally, we need to position constrain the original IK effector to the soft effector, so that it follows it. Otherwise, all that would happen is that the length of the chain would change.

So that’s it, we’re there. All that remains is to package it all up into a plugin with a handy custom property to drive it all. This is what the new property page looks like:

The bone lengths can be adjusted (and even animated) from this PPG, as well as the Soft Distance parameter, which behaves in exactly the same way as in the non-stretchy Soft IK. The bone lengths are all driven from this PPG using expressions. Note, that for extra performance we only have one Scripted Operator here which operates on the BoneScale parameter of the PPG.

I’ve updated the original Soft Effector script to include a new menu item to apply the Stretchy Bone Soft IK system to any chain you select.

Soft Effector Download (3Kb)

Incidentally, if you ever want to remove the Soft IK (stretchy or otherwise) from a bone chain. All you have to do is delete the Soft Effector object. All the expressions and operators will be automatically deleted since they will have lost their connections. Nice and tidy!

P.S. Thanks to Daniele Niero for suggesting I cache the math objects inside the operator user data!

In XSI, we need a way of allowing the chain to gradually fall behind the position of the effector. This stops the effector from snapping the chain into the final position, and gives rise to a much more natural motion (as seen below).

(Note: If you are unable to view the GIF animation, you may need to change settings in your Firewall. For example, in Zone Alarm, you need to disable Privacy->Ad Blocking)

The snap effect visible in the top chain isn’t due to some inaccuracy in the IK calculation, but is just down to a simple geometrical effect. It can be shown mathematically (see diagram below) that as the effector pulls the chain into the final position, the velocity of the bones towards their final position tends towards infinity! This is obviously not desirable and can make the animation look jerky and artificial.


(click to enlarge)

To achieve the type of motion shown in the lower chain, we need to add an extra object which is used to drive the IK chain effector. A scripted operator links the two objects and we use a mathematical expression to give the desired result. The only bits of information the operator needs are the total chain length, and something I call the “Soft Distance”. This represents the distance from the chain’s full extension that the effect starts to work.

There is one downside with this approach, as it means you can’t fix the end point of the chain in position. However, this can be solved as long as you are willing to put a small amount of extra work to animate the Soft Distance parameter.

The soft effector object has a custom property to allow you to easily control the operator.

To make things easier for you, the Chain Length is calculated by an expression which sums the length of each bone in the chain. This allows you to change the length of a bone, and still have the Soft IK work.

In the diagram below, the Soft Distance is represented by the distance between the green and the red lines (actually, they’re circles, but too large to see). As soon as the effector reaches the green line, our scripted operator starts to lag the chain effector behind our soft effector. This is done while making sure that we have no jerkiness in the motion of the chain effector.

(Note: If you are unable to view the GIF animation, you may need to change settings in your Firewall. For example, in Zone Alarm, you need to disable Privacy->Ad Blocking)

The equation used to create this effect is shown below:

(click to enlarge)

For a chain length of 3 and a soft distance of 1, this equation looks something like this:

(click to enlarge)

You can use the following link to download the Soft Effector plugin:

Soft Effector Download (3 Kb)

(Note that this also features the Stretchy Bone Chain extension shown in Part 2 of this article)

To install the plugin, just download the zip file, unzip the script, and copy it to a “Plugins” folder. So for example, you can use your “C:\users\username\Softimage\XSI\Application\Plugins” folder, or one in your workgroup.

To use the Soft Effector, create a 2D or 3D Skeleton Chain, select any part of the chain, and go to “Model->Create->Skeleton->Apply Soft Effector”.