The conversation that has taken place in the replies to this article are important in defining how the article’s information should be applied, however, you probably lost most beginners and a good portion of everyone else by the end of it ;). This article is great (with a few disclaimers warning about assumed usage)!

As someone already mentioned, readability should win the day when speed of various methods is either too close to worry about, or simply unnecessary. I would suggest creating good, clean, easy to read, and edit, code that works first, then optimize where necessary. Especially when working with a group of people that have to work with your code…and especially when working to a deadline. In production a developer can’t always write awesome code just to stroke their “ego”, you have to spend the time where the production will best benefit. This is a hard-won lesson and can apply to anything from writing code to animating.

In python, I always reach for the object iterating for-loop. I find range-based looping is rarely something I will use by choice (in JScript it was the primary looping method, but while learning Python I read a book which said I would find that I simply would’t need it most of the time. I have found this to be true). I would only move to other methods of looping to address performance issues. This is where this article gives some great clues to new XSI/Python users.

Nice one!

- Rafe

while xsi almost always returns tuples when dealing with returning arrays to the user, there are a lot of instances where you could be working with external data as well, and that could very well return a list right away.

this example aside though, often one ends up applying these techniques and tricks to custom data that won”t necessary be a tuple, nor easily pre-determined in its layout (IE: especially when working with databases and walking one you never know with how many branching offs you will end up with in some searches), but that is probably out of the scope of the article.

casting a tuple to a list by list() or creating a pre-determined sized list through list comprehension will be nearly instantaneous; and then you can modify it in place.
the speed boost will still be considerable over creting the list by processing and appending the entry at every cycle.

Also, if you multiply a None in a list I think that triggers a python optimization to generate a presized list even faster, something like myList = [None]*Size, but don”t quote me on this one as I don”t remember where or when I read it, and using list comp is usually so fast that I never concerned myself with finding a better way to pre-size a list.

[Bernard] I think you answered my qestion, I was interested in the time curve on your side of things. Thanks for that.

>> (no need to swear on what the purpose is, as I said the point was never to try and belittle you, otherwise I would have brought forward the subject of poutine, not mapping functions).

[Bernard] mmmm poutine…. What a fascinating subject, isn”t it? That said, I can hardly see how you could belittle me (or a Canadian) on such a subject. ;)

>> IE: manipulating in place can be a lot faster then creating by appending while the loop runs, but you won’t always be able to pre-determine the size of the list in all dimensions (especially for very generically purposed pieces of code), or it would need counter-productive procedures or just too much time to test it in all situations.

Okay here I have a question for you. Unless I have missed something, there is no way to specify a list length in Python like in VBScript or JScript. On the other hand, the tuple has a fixed length, and its elements can”t be modified in-place. So I”m not sure what you meant with that statement…… if you are dealing with a list, then its length doesn”t really matter, but on the other hand, if you are dealing with a tuple, then you have no choice but to make copies. Correct?

>> I’ll be happy to acknowledge the last paragraph of the initial post in regards to my comment and consider it crossed out.

[Bernard] Thank you.

>> does this make my initial point any clearer or is there anything I left out or mis-interpreted/mis-presented.

Fair enough, I think we have reached a consensus and finally seem to be talking the same language by now.

Cheers
Bernard

in other cases, especially on much more complex operations then a simple addition, the difference map can make will go up tenfold, and we”ve had cases where just mapping a function, with no other optimization, boosted a snippet’’s speed by well over x10.

in some other cases it wouldn”t make much of a difference, and in some (although seldomly encountered) others a lambda could even slow down things a bit (the peculiarity of lambda functions is the way the process is mapped ahead of execution, and in cases where this process is voided, or not applied to a large enough table, it can be counterproductive).

Different techniques will require different steps.

IE: manipulating in place can be a lot faster then creating by appending while the loop runs, but you won”t always be able to pre-determine the size of the list in all dimensions (especially for very generically purposed pieces of code), or it would need counter-productive procedures or just too much time to test it in all situations.

also, if you were to take away from the equation the time needed to create or manipulate lists and other elements which differ from technique to technique, results would change again, and while knowing the performance of each part of the execution can be relevant, in the end it’’s the performance of the whole, possibly in situations that are real-world related, that matters.

this is, obviously, neglecting other important elements of software design, but for sheer speed based operations (like manipulating geometry) the self-contained example is quite pertinent.

I”ll be happy to acknowledge the last paragraph of the initial post in regards to my comment and consider it crossed out.
does this make my initial point any clearer or is there anything I left out or mis-interpreted/mis-presented.

cheers.
Raff

#INFO : for range appending the new value 0.383707485695
#INFO : check
#INFO : for each element and appending 0.321049263254
#INFO : check
#INFO : for range modifying in place 0.337336047004
#INFO : check
#INFO : mapping the function 0.170906125977
#INFO : check
#INFO : mapping a lambda function 0.147533536157
#INFO : check
#INFO : list comprehension 0.222757300532
#INFO : check
#INFO : list comprehension with embedded function 0.11899168944
#INFO : check
#INFO : total time elapsed: 2.75101092793

>> I said the way your statements try to encompass the whole looping techniques thing is what I’m arguing.

I would like to point you to the last paragraph of my initial post. But I take your point nonetheless.

Thanks for the code, I appreciate.

Bernard

import win32com
import time

def simpleAddition(foo):
    foo = foo +1
    return foo

#Application.CreatePrim("Sphere", "MeshSurface", "", "")
#Application.SetValue("sphere.polymsh.geom.subdivu", 300, "")
#Application.SetValue("sphere.polymsh.geom.subdivv", 300, "")

overallStartTime = time.clock()

tupPointsPosition = Application.Selection(0).ActivePrimitive.Geometry.Points.PositionArray

startTime = time.clock()
forRangeAppendPointsPosition = [[],[],[]]
for i in range(len(tupPointsPosition[0])):
    forRangeAppendPointsPosition[0].append(simpleAddition(tupPointsPosition[0][i]))
    forRangeAppendPointsPosition[1].append(simpleAddition(tupPointsPosition[1][i]))
    forRangeAppendPointsPosition[2].append(simpleAddition(tupPointsPosition[2][i]))
endTime = time.clock()
print "for range appending the new value " + str(endTime-startTime)
testList = forRangeAppendPointsPosition
if testList == forRangeAppendPointsPosition: print"check"

startTime = time.clock()
forInPointPosition = [[],[],[]]
for i in tupPointsPosition[0]:
    forInPointPosition[0].append(simpleAddition(i))
for i in tupPointsPosition[1]:
    forInPointPosition[1].append(simpleAddition(i))
for i in tupPointsPosition[2]:
    forInPointPosition[2].append(simpleAddition(i))
endTime = time.clock()
print "for each element and appending " + str(endTime-startTime)
if testList == forInPointPosition: print"check"

startTime = time.clock()
forRangeInPlacePointsPosition = [list(tupPointsPosition[0]),list(tupPointsPosition[1]),list(tupPointsPosition[2])]
for i in range(len(tupPointsPosition[0])):
    forRangeInPlacePointsPosition[0][i] = simpleAddition(tupPointsPosition[0][i])
    forRangeInPlacePointsPosition[1][i] = simpleAddition(tupPointsPosition[1][i])
    forRangeInPlacePointsPosition[2][i] = simpleAddition(tupPointsPosition[2][i])
endTime = time.clock()
print "for range modifying in place " + str(endTime-startTime)
if testList == forRangeInPlacePointsPosition: print"check"

#no for each modifying in place as retrieving the index of the value
#makes modification in place unreliable and dangerous
#or overly complicated and slow to make it safe

startTime = time.clock()
mapPosX = map(simpleAddition, tupPointsPosition[0])
mapPosY = map(simpleAddition, tupPointsPosition[1])
mapPosZ = map(simpleAddition, tupPointsPosition[2])
mapPointsPosition = [mapPosX, mapPosY, mapPosZ ]
endTime = time.clock()
print "mapping the function " + str(endTime-startTime)
if testList == mapPointsPosition: print"check"

startTime = time.clock()
mapLambdaPosX = map(lambda x: x+1, tupPointsPosition[0])
mapLambdaPosY = map(lambda x: x+1, tupPointsPosition[1])
mapLambdaPosZ = map(lambda x: x+1, tupPointsPosition[2])
mapLambdaPointsPosition = [mapPosX, mapPosY, mapPosZ ]
endTime = time.clock()
print "mapping a lambda function " + str(endTime-startTime)
if testList == mapLambdaPointsPosition: print"check"

startTime = time.clock()
listCompPointsPosition = [[simpleAddition(i) for i in tupPointsPosition[0]],[simpleAddition(i) for i in tupPointsPosition[1]],[simpleAddition(i) for i in tupPointsPosition[2]]]
endTime = time.clock()
print "list comprehension " + str(endTime-startTime)
if testList == listCompPointsPosition: print"check"

startTime = time.clock()
listCompDirectPointsPosition = [[i+1 for i in tupPointsPosition[0]],[i+1 for i in tupPointsPosition[1]],[i+1 for i in tupPointsPosition[2]]]
endTime = time.clock()
print "list comprehension with embedded function " + str(endTime-startTime)
if testList == listCompDirectPointsPosition: print"check"

overallEndTime = time.clock()
print "total time elapsed: " + str(overallEndTime-overallStartTime)

I said the way your statements try to encompass the whole looping techniques thing is what I”m arguing.

remember when I said that you should try different testbeds? and that applying mathematical operations to an array will be the fastest by mapping or comprehension and that such a situation wasn”t so uncommon (it’’s infact more performance critical then walking selections or curves, as such topology walks are at the heart of deformers, which NEED to be fast).

in the following code, which is representative of the time required to create lists or not in each technique and how they are created, you can see the speed progression I described in my reply.
it is just one of MANY examples where performance ladders are considerably different from that in your conclusions.

the innner working of an interpreter and the various caching/loading/reloading and instancing operations needed to run them can tip the scales in considerably different ways in different situations.
IE: notice how a lambda function will make map slightly faster, but will slow down list comprehension considerably.

if the site kills the formatting, indent once lines:
5, 6, 19, 20, 21, 30, 32, 34, 43, 44, 45
I hope it helps furthering your understanding of programming and improving your scripts:

——
Editor’’s note:

Script removed.
Please refer to the script in this comment.
——

It is with the greatest of interest that I have read your last post. Recently I had some time to tinker about it, so let’’s review your comments…

>> you first have to do it for XSI’s SDK, figuring out what objects and methods, under a constant condition (like using always the same iteration technique), offer what kind of results.

[Bernard] This article is about using in XSI some of the available looping techniques offered by Python, with the only exception being FindObjects(), wich I provided only for a matter of comparison. I don”t know where you got that but I have never pretended that some techniques were better or worse than some techniques offered by the XSI objects and methods (other than the FindObjects call). I voluntarily left out things like FindChildren, Filter, Find, Item, and so on. I focused entirely on Python specific looping techniques, hence the title of this article. The intention was not to cover all possible combinations of all available looping techniques, wich would probably make a book. I don”t see how this made my conclusions flawed.

>> once you’ve done that you test the same set under all the looping techniques.

[Bernard] This is what I have done nearly throughout the entire article, Raffaele…

>> once you’ve done that you test this second level permutation (which will be at least 16 different snippets just using 4 possible looping techniques on 4 different kinds of storing and handling data) you’ll then have to figure out what direct optimization techniques you could use and when they are beneficial or detrimental.

[Bernard] Okay another point here. I”m in complete agreement with you that each looping technique has to be carefully chosen depending on the context. Yes, in some cases, some types of loops will work better than others. I used only a handful of looping techniques in only a handful of contexts.

Still, one has to admit that even though pulling constants out of a few tests is not possible, tendencies, or should I say, outlines, can be drawn and hold on tight until there is proof of the contrary is found. As opposed to what you state I have not attempted to find stone truths, I explored few possibilities in few context. The possiblities I have explored may or may not extend beyond their context, that is up again for experimentation. Still, I believe the conclusion I have pulled from this experimentation is 100% valid and could easily apply to many different scenarios.

>> one example is map, that you insist saying isn’t much of a performance booster when dealing with functions written inside your python code. that statement could be almost correct in this one particular case, but have you tryied mapping a simple transform over a position array list?

[Bernard] What if I did?

===================================================
1- Moving a keyframe in X translation by 3 units

In this exercice, I have created a null, animated it over 100 frames, changed the fcurves to linear, then plotted its transformations for each frame. As a result, the null had 100 keyframes on all translation axes.
The script has to take the posx fcurve, and apply a simple addition of 3 units to each keyframe.

import win32com
import time

c = win32com.client.constants
xsi = Application

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
xsi.logmessage( round( time2 – time1, 3 ) )

def moveUp( oKey ):
# Append in-place the list of new keyframes
aList.append( oKey.time )
aList.append( oKey.value + nAmount )

def main():

global aList
global nAmount

# Get selected object
oSel = xsi.selection(0)

# Specify the amount of transformation we want
nAmount = 3.0

# Create empty list to store new array of time/value pairs
aList = []

# Map keys to function
map( moveUp, oSel.posx.source.keys )

# Put new keyframes onto fcurve
oSel.posx.source.setkeys( aList )

# Call timer
timer()

# RESULTS:
#INFO :
#INFO : 0.052
#INFO : 0.104
#INFO : 0.156
#INFO : 0.209
#INFO : 0.26
#INFO : 0.313
#INFO : 0.365
#INFO : 0.417
#INFO : 0.469
#INFO : 0.521

In the next script, I just made a slight modification. Instead of putting times and values into a list and then setting keys on the fcurve, I modify the keys in-place directly. That made the code a little bit shorter, as there was no need to declare globals.

import win32com
import time

c = win32com.client.constants
xsi = Application

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
xsi.logmessage( round( time2 – time1, 3 ) )

def moveUp( oKey ):
oKey.value += 3.0

def main():

# Get selected object
oSel = xsi.selection(0)

# Map keys to function
map( moveUp, oSel.posx.source.keys )

# Call timer
timer()

# RESULTS:
#INFO :
#INFO : 0.061
#INFO : 0.122
#INFO : 0.183
#INFO : 0.245
#INFO : 0.307
#INFO : 0.368
#INFO : 0.429
#INFO : 0.491
#INFO : 0.552
#INFO : 0.614

Now let’’s try with a classic for loop. There was no need for another function, now the loop can be done in the main function.

import win32com
import time

c = win32com.client.constants
xsi = Application

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
xsi.logmessage( round( time2 – time1, 3 ) )

def main():

# Get selected object
oSel = xsi.selection(0)
oFcurve = oSel.posx.source

# Specify the amount of transformation we want
nAmount = 3.0

# Create empty list to store new array of time/value pairs
aList = []

for oKey in oFcurve.keys:
aList.append( oKey.time )
aList.append( oKey.value + nAmount )

# Put new keyframes onto fcurve
oFcurve.setkeys( aList )

# Call timer
timer()

# RESULTS
#INFO :
#INFO : 0.051
#INFO : 0.103
#INFO : 0.155
#INFO : 0.206
#INFO : 0.258
#INFO : 0.309
#INFO : 0.36
#INFO : 0.412
#INFO : 0.464
#INFO : 0.515

Very, very slightly faster…. It’’s certainly not excluded that I have not written the best possible way the map() calls, but there are also other factors to consider. First, since the loop is done in the main function, the calls to another are suppressed. In Python, every call, as lightweight as it might be, involves a performance cost. Also, the fact that I have suppressed the need to lookup the global scope for the aList and nAmount objects might have speed up things a little, as I already explained that looking up a wider scope than the “current” one also involves a performance cost.

Now let’’s try a modification in-place using a classic for loop:

import win32com
import time

c = win32com.client.constants
xsi = Application

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
xsi.logmessage( round( time2 – time1, 3 ) )

def main():

# Get selected object
oSel = xsi.selection(0)
oFcurve = oSel.posx.source

# Specify the amount of transformation we want
nAmount = 3.0

# Create empty list to store new array of time/value pairs
aList = []

for oKey in oFcurve.keys:
oKey.value += nAmount

# Call timer
timer()

# RESULTS
#INFO :
#INFO : 0.06
#INFO : 0.122
#INFO : 0.183
#INFO : 0.245
#INFO : 0.307
#INFO : 0.368
#INFO : 0.43
#INFO : 0.492
#INFO : 0.554
#INFO : 0.617

Interesting…. Compared to the map version, the for loop seems faster in the beginning, but is gradually slowing down, to the point where it is slower than the map() version. What is also interesting is the fact that the population of the list in-place was, both with map and for, rather steady compared to each other.

Now I wonder what would be the result of the last two scripts, those that modify keyframe values in place, if we had 10,000 keyframes instead of 100. Let see.

The map version:

#INFO :
#INFO : 64.185
#INFO : 128.029
#INFO : 194.52
#INFO : 258.883
#INFO : 326.49
#INFO : 392.737
#INFO : 459.738
#INFO : 526.23
#INFO : 593.348
#INFO : 660.818

The for version:

#INFO :
#INFO : 61.918
#INFO : 123.811
#INFO : 185.709
#INFO : 247.661
#INFO : 309.602
#INFO : 371.551
#INFO : 433.595
#INFO : 495.631
#INFO : 557.791
#INFO : 619.819

Hummm in this case, the map version not only did worse than the for version, but the result is not the same at all as was we had before, the speed of the for loop remains more consistantly lower than the map version.

===================================================
2- Computing tranform multiplications

Very well, enough of that. I figured that I could do a different exercice. Perhaps I have not chosen the right use for the map call. Thus I have created a null that I keyframed at the origin. Then I moved it to (1000, 1000, 1000) and keyframed it. Then I changed the curves to linear interpolation. Then I duplicated from animation this null, I ended up with 1000 nulls. In this exercice I extract the transformations of the nulls, multiply them by another transform, and set these transforms back on the nulls.

import win32com
import time

c = win32com.client.constants
xsi = Application
XSIMath = win32com.client.Dispatch( ”XSI.Math” )

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
#xsi.logmessage( round( time2 – time1, 3 ) )

def moveDown( oObject ):
# Get current object transforms
oObjTransform = oObject.Kinematics.Local.Transform

# Create new transformation
oNewTransform = XSIMath.CreateTransform()

# Multiply object transform by global transform
oNewTransform.Mul( oObjTransform, oTransform )

oObject.Kinematics.Local.Transform = oNewTransform

def main():

global oTransform
oTransform = XSIMath.CreateTransform()
oTransform.SetTranslationFromValues( -1.578, 3.45987, 4.23490 )

map( moveDown, xsi.selection )

# Call timer
timer()

# RESULTS
#INFO :
#INFO : 2.916
#INFO : 5.857
#INFO : 8.809
#INFO : 11.765
#INFO : 14.708
#INFO : 17.669
#INFO : 20.625
#INFO : 23.542
#INFO : 26.487
#INFO : 29.444

Then with a for loop version:

import win32com
import time

c = win32com.client.constants
xsi = Application
XSIMath = win32com.client.Dispatch( ”XSI.Math” )

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
xsi.logmessage( round( time2 – time1, 3 ) )

def main():

oTransform = XSIMath.CreateTransform()
oTransform.SetTranslationFromValues( -1.578, 3.45987, 4.23490 )

for oObject in xsi.selection:
# Get current object transforms
oObjTransform = oObject.Kinematics.Local.Transform

# Create new transformation
oNewTransform = XSIMath.CreateTransform()

# Multiply object transform by global transform
oNewTransform.Mul( oObjTransform, oTransform )

oObject.Kinematics.Local.Transform = oNewTransform

# Call timer
timer()

#INFO :
#INFO : 2.912
#INFO : 5.876
#INFO : 8.869
#INFO : 11.851
#INFO : 14.87
#INFO : 17.857
#INFO : 20.848
#INFO : 23.821
#INFO : 26.813
#INFO : 29.786

Mmmm not much differences, all in all the for is slower by a fraction. Not very conclusive. Let’’s make some modifications to the scripts. This time, instead of creating local transform objects in every iteration, we”ll create a reusable transform object. I”m not sure it will speed up or slow things down, but it worth a try, no?

import win32com
import time

c = win32com.client.constants
xsi = Application
XSIMath = win32com.client.Dispatch( ”XSI.Math” )

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
xsi.logmessage( round( time2 – time1, 3 ) )

def moveDown( oObject ):
# Get current object transforms
oObjTransform = oObject.Kinematics.Local.Transform

# Multiply object transform by global transform
oNewTransform.Mul( oObjTransform, oTransform )

oObject.Kinematics.Local.Transform = oNewTransform

def main():

global oTransform
oTransform = XSIMath.CreateTransform()
oTransform.SetTranslationFromValues( -1.578, 3.45987, 4.23490 )

global oNewTransform
oNewTransform = XSIMath.CreateTransform()

map( moveDown, xsi.selection )

# Call timer
timer()

# RESULTS
#INFO :
#INFO : 2.186
#INFO : 4.45
#INFO : 6.734
#INFO : 8.993
#INFO : 11.249
#INFO : 13.496
#INFO : 15.743
#INFO : 18.012
#INFO : 20.298
#INFO : 22.537

Nice! Just using a global object instead of recreating it at each iteration made things faster. Now let see how things are going with the for loop.

import win32com
import time

c = win32com.client.constants
xsi = Application
XSIMath = win32com.client.Dispatch( ”XSI.Math” )

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
xsi.logmessage( round( time2 – time1, 3 ) )

def main():

oTransform = XSIMath.CreateTransform()
oTransform.SetTranslationFromValues( -1.578, 3.45987, 4.23490 )

# Create new transformation
oNewTransform = XSIMath.CreateTransform()

for oObject in xsi.selection:
# Get current object transforms
oObjTransform = oObject.Kinematics.Local.Transform

# Multiply object transform by global transform
oNewTransform.Mul( oObjTransform, oTransform )

oObject.Kinematics.Local.Transform = oNewTransform

# Call timer
timer()

# RESULTS
#INFO :
#INFO : 2.224
#INFO : 4.492
#INFO : 6.705
#INFO : 8.975
#INFO : 11.209
#INFO : 13.437
#INFO : 15.658
#INFO : 17.871
#INFO : 20.095
#INFO : 22.351

The for loop starts a little slower, but catches and even surpasses the map loop.

Okay, time to pull out some conclusions. First, there are multiple other uses of loops. These tests were just another attempt at measuring differences in speed in a given context. The difference were generally very minor, by a magnitude of 100ths of a second for the “quick” loops, and by an order of 3 to 40 seconds for the longer loops. In some cases the map performed better, in other case the for loop performed better. Only in the longer loops did the for really provided a big difference. So based on this new series of tests, I still can”t change my mind about the fact that a for loop is faster than a map when used with Python functions.

===================================================
3- Other forms of loop

But wait! I remembered that lambdas and list comprehensions were mentioned. Lambdas are still a little bit obscure to me, but I decided to give it a go, hoping I will not mess things. Let see….

import win32com
import time

c = win32com.client.constants
xsi = Application

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
xsi.logmessage( round( time2 – time1, 3 ) )

def main():

# Get selected object
oSel = xsi.selection(0)
oFcurve = oSel.posx.source

# Specify the amount of transformation we want
nAmount = 3.0

# Map to lambda function, returns tuples
aTuples = map( ( lambda oKey: ( oKey.time, oKey.value + nAmount ) ), oFcurve.keys )

# Flatten list
aFlatList = [ nValue for tTuple in aTuples for nValue in tTuple ]

# Put new keyframes onto fcurve
oFcurve.setkeys( aFlatList )

# Call timer
timer()

# RESULTS
#INFO :
#INFO : 0.052
#INFO : 0.104
#INFO : 0.156
#INFO : 0.207
#INFO : 0.259
#INFO : 0.31
#INFO : 0.362
#INFO : 0.414
#INFO : 0.465
#INFO : 0.517

I suspect that I might have not the most perfect use of the lambda (like, I don”t know if it’’s possible to return multiple values without a tuple), but still it is a perfectly legal use of lambda to use it in a map call. Possibly the list comprehension right after the map/lambda that flattens the values may involve a performance hit that nullifies the use of the map/lambda combination.

Now let’’s try with a list comprehension version…

import win32com
import time

c = win32com.client.constants
xsi = Application

def timer():

xsi.logmessage( ”” )

# Record current time
time1 = time.clock()

# Start loop to perform single function calls
for i in range( 0, 10 ):

# Call function
main()

# Record new time
time2 = time.clock()

# Print execution time, rounded to 3 miliseconds
xsi.logmessage( round( time2 – time1, 3 ) )

def main():

# Get selected object
oSel = xsi.selection(0)
oFcurve = oSel.posx.source

# Specify the amount of transformation we want
nAmount = 3.0

# Get pairs of new time/value
aTuples = [ ( oKey.time, oKey.value + nAmount ) for oKey in oFcurve.keys ]

# Flatten list
aFlatList = [ nValue for tTuple in aTuples for nValue in tTuple ]

# Put new keyframes onto fcurve
oFcurve.setkeys( aFlatList )

# Call timer
timer()

# RESULT
#INFO :
#INFO : 0.051
#INFO : 0.104
#INFO : 0.156
#INFO : 0.208
#INFO : 0.26
#INFO : 0.311
#INFO : 0.363
#INFO : 0.414
#INFO : 0.465
#INFO : 0.516

Again, extremely minimal difference….

===================================================
4- Summary of tests

Okay so let’’s put all the result tables next to each other, so we can have a better idea….

#INFO :
#INFO : 0.052
#INFO : 0.104
#INFO : 0.156
#INFO : 0.209
#INFO : 0.26
#INFO : 0.313
#INFO : 0.365
#INFO : 0.417
#INFO : 0.469
#INFO : 0.521

#INFO :
#INFO : 0.051
#INFO : 0.103
#INFO : 0.155
#INFO : 0.206
#INFO : 0.258
#INFO : 0.309
#INFO : 0.36
#INFO : 0.412
#INFO : 0.464
#INFO : 0.515

#INFO :
#INFO : 0.052
#INFO : 0.104
#INFO : 0.156
#INFO : 0.207
#INFO : 0.259
#INFO : 0.31
#INFO : 0.362
#INFO : 0.414
#INFO : 0.465
#INFO : 0.517

#INFO :
#INFO : 0.051
#INFO : 0.104
#INFO : 0.156
#INFO : 0.208
#INFO : 0.26
#INFO : 0.311
#INFO : 0.363
#INFO : 0.414
#INFO : 0.465
#INFO : 0.516

# 100 keyframes
#INFO :
#INFO : 0.061
#INFO : 0.122
#INFO : 0.183
#INFO : 0.245
#INFO : 0.307
#INFO : 0.368
#INFO : 0.429
#INFO : 0.491
#INFO : 0.552
#INFO : 0.614

#INFO :
#INFO : 0.06
#INFO : 0.122
#INFO : 0.183
#INFO : 0.245
#INFO : 0.307
#INFO : 0.368
#INFO : 0.43
#INFO : 0.492
#INFO : 0.554
#INFO : 0.617

# 10,000 keyframes
#INFO :
#INFO : 64.185
#INFO : 128.029
#INFO : 194.52
#INFO : 258.883
#INFO : 326.49
#INFO : 392.737
#INFO : 459.738
#INFO : 526.23
#INFO : 593.348
#INFO : 660.818

#INFO :
#INFO : 61.918
#INFO : 123.811
#INFO : 185.709
#INFO : 247.661
#INFO : 309.602
#INFO : 371.551
#INFO : 433.595
#INFO : 495.631
#INFO : 557.791
#INFO : 619.819

#INFO :
#INFO : 2.916
#INFO : 5.857
#INFO : 8.809
#INFO : 11.765
#INFO : 14.708
#INFO : 17.669
#INFO : 20.625
#INFO : 23.542
#INFO : 26.487
#INFO : 29.444

#INFO :
#INFO : 2.912
#INFO : 5.876
#INFO : 8.869
#INFO : 11.851
#INFO : 14.87
#INFO : 17.857
#INFO : 20.848
#INFO : 23.821
#INFO : 26.813
#INFO : 29.786

#INFO :
#INFO : 2.186
#INFO : 4.45
#INFO : 6.734
#INFO : 8.993
#INFO : 11.249
#INFO : 13.496
#INFO : 15.743
#INFO : 18.012
#INFO : 20.298
#INFO : 22.537

#INFO :
#INFO : 2.224
#INFO : 4.492
#INFO : 6.705
#INFO : 8.975
#INFO : 11.209
#INFO : 13.437
#INFO : 15.658
#INFO : 17.871
#INFO : 20.095
#INFO : 22.351

To conclude this series of tests, I will say that the use I made of map, for, lambda and list comprehension did NOT invalidated any of the conclusion from the top article, and I have yet to find a scenario where using map will be faster than for loops with Python functions. At this point, if you are in disagreement with me, I think it would be a better solution for you to provide some code. Unless you provide evidence of your statements, I”m affraid I”ll have to stay on my positions on that one. I think I have done my homeworks.

Btw, all scripts I have wrote for this post are available here:
http://www.bernardlebel.com/scripts/xsi/scripts/others/xsiblog/pythonloopingtechniques/

>> when you use a single test bed for your performance analisys, and one that involves such a highly SDK dependant thing like XSI’s internal collections, you’re totally voiding your efforts.

[Bernard] With all due respect to your experience and knowledge, I”m in largely in disagreement here. Again, this article is all about using Python looping techniques in XSI, *nothing more*. The code runs in XSI at all time, in the same build, same machine, same OS, same Python version, well same conditions at all times. In this article I have questioned Python *in XSI*, period. Of course it’’s dependant of the SDK. So? That means we should try to benchmark Python’’s performance? Of course the results may be different than running similar code in another environment. But I did not talked about that.

One last thing…

>> hopefully now you understand what I meant when I said I find the article flawed in its approach and in the adamantine quality some of the statements have in them.

I take what I do very seriously, and a considerable amount of time has been spent on these posts. Therefore, I consider that it would be the most elementary courtesy and respect from you, if you wish to further refute my statements, to show a similar commitment and provide tangible material to support your statements. I think I have gone far enough with this for you to go past the commenting approach and take an educating one.

I know you are a knowledgeable guy and I respect that, and would be grateful if you could teach me something; I”m happy to revisit my positions, as long as we”re on the same level of seriousness. Still I thank you for your comments, they made me learn a few things already.

Bernard

what you deal with in this article is clearly oriented to performance tuning.
now, performance tuning is a VERY task specific issue to deal with, no 2 pieces of code, even when the apparent differences are minimal, benefit in the same way from the same exact tuning in the same way.

performance is dictated by several elements working in sinergy. part of it is the language you are using or, more specifically, the backend to that language and how it deals with different programming paradigms and the various and different tools that it offers to handle a generic dataset in substantially different ways. however, a HUGE part of performance issues could very well spawn from something entirely different, like the API and the SDK you are using, the way such tools handle calls marshalling and internal automation/optimization of processes and all that.

when you use a single test bed for your performance analisys, and one that involves such a highly SDK dependant thing like XSI’’s internal collections, you”re totally voiding your efforts.

one example is map, that you insist saying isn”t much of a performance booster when dealing with functions written inside your python code. that statement could be almost correct in this one particular case, but have you tryied mapping a simple transform over a position array list?

on something simple, like getting the position array, decomposing it and adding a fixed amount to one axis of the position, and then dumping it back into the geometry, mapping that function VS looping it in a range provides a tenfold difference in performance, and that is because what you do is creating the list, manipulating it all in one go, and then dump it back.
further then that, in such a simple case you could achieve the same result, without obfuscating the code, and inlining the operation with a lambda, and that would give you another small, but nearly irrelevant boost.
last but not least, the same thing, with list comprehension, will probably result in something even faster, if only by 10-20%, then map.

now, isn”t that a case that happens VERY often when writing for XSI? manipulating an array to alter geometry I mean, and isn”t that giving results radically different from what you are stating as a fairly cold hard truth here?

trying to generalize the subject of performance tuning, even in such a simple environment like an interpret language in one single application, is still damn hard.
there’’s hundreds of pages in huge very technical books that deal with optimization in a single language in a single API in a single environment (like optimizing terrain generation in C + OGL), and yet they barely scratch the surface.

drawing conclusions about all the possibilities in python from one single test is misleading to say the least.

if you want to decompose and chart performance optimization, you first have to do it for XSI’’s SDK, figuring out what objects and methods, under a constant condition (like using always the same iteration technique), offer what kind of results.
once you”ve done that you test the same set under all the looping techniques.
once you”ve done that you test this second level permutation (which will be at least 16 different snippets just using 4 possible looping techniques on 4 different kinds of storing and handling data) you”ll then have to figure out what direct optimization techniques you could use and when they are beneficial or detrimental.

like sometimes committing a collection to a different datatype, handling it, and committing it back into a collection (or parsing the dataset) will provide obvious boosts, because the translation overhead is only a small loss compare to the speed boost gained from a language’’s tricks you could use.
some other times that overhead will negate the benefits and then slow you down some.

considering 4 or 5 different tricks and techniques of committing or dereferencing data, your 3rd level permutation would be already 80 different situations to test and chart, and that is leaving out all the shades of gray inbetween.

hopefully now you understand what I meant when I said I find the article flawed in its approach and in the adamantine quality some of the statements have in them.