Re: Workgroup Shaders and Batchserve

Date : Thu, 11 May 2006 21:14:02 -0700
To : XSI(at)Softimage.COM
From : Andy Jones <andy(at)thefront.com>
Subject : Re: Workgroup Shaders and Batchserve
I agree with Bernard, even just on principal. I never even bothered to try a shared workgroup. In general, with rendering, there are good, efficient ways to do things and bad, inefficient ways of doing things. Leaving files that get read repeatedly by render nodes in a shared location is typically a bad, inefficient way to do things. Textures are another obvious example -- .map especially. I would only recommend a shared workgroup location to someone who doesn't have the time, neccessity, or know-how to do things better. That probably includes a majority of users out there, but they should be aware that their network will be taking a hit as they expand.

Currently, our nodes do an incremental sync of their workgroups just before BatchServe launches XSIBatch. That way, they're always on the latest commited workgroup before they start a render. Since we're constantly in active development for our workgroup, and I want the ability to recompile shaders in the middle of a job if need be, this has been a good way to go. We have yet to have a single instance of a renderfarm machine running into registry problems with shaders, though I have seen it once or twice on workstations. Fortunately, we haven't seen the unretained workgroup path issue. Maybe a Linux thing?

Byron, it sounds very possible that the registry problem is what's going on here. Or, this is a stupid question, but is there any chance BatchServe is launching the wrong version of XSI? Do the machines have XSI 5.0 or earlier installed on them? My confidence in XSI 5.1's system of dealing with unregistered shaders was severely shaken by the breaking of the ConnectByProgID method. Also, if it's failing to load the sources, I don't think it's a missing .dll problem. That would give you a mental ray error, not an XSI error. You should also check the user of the BatchServe service in the Windows Management console, as Dana suggested, to make sure it's using the right user. Or, just check the user preference sets on the machine to see if there's an incorrect set of preferences for an unexpected user.

-Andy

Bernard Lebel wrote:

What do you consider "really large" as opposed to "small to medium"?

Here, we have 45 render nodes, which I consider to be a medium setup.
We use only 4 custom shaders, and actually only 2 of them are used
very often. I have made the test many number of times (and would let
the test run for several days), as soon as the custom shaders are read
from a shared network location, the network is less responsive. I mean
not a matter of a few 1/100s of a second, I mean that everyone would
scream that the network is slow right after I enabled that setup.
Browsing, flipping and listing directories would have a humanly
mesurable lag. As soon as I go back to a local setup, things would
revert to normal (and people would calm down!)

There is of course nothing scientifical in those tests, but having
done it several times and without any other changes to our pipeline,
here and at another company I worked for, kind of consolidated the
belief that even with a reasonably small number of nodes (20-50) and
custom shaders (2-5), a shared shader location is a rather significant
constraint on the network (depending on the the quality of the network
setup, that is). If we didn't have that problem with unretained
workgroup path I would definitely consider copying workgroups locally.


My two cents Bernard


On 5/11/06, kim aldis <kim(at)aldis.org.uk> wrote:



> -----Original Message-----
> From: owner-xsi(at)Softimage.COM
> [mailto:owner-xsi(at)Softimage.COM] On Behalf Of Bernard Lebel
> Sent: 11-May-2006 22:07
> To: XSI(at)Softimage.COM
> Subject: Re: Workgroup Shaders and Batchserve
>
> On 5/11/06, Byron Nash <byronnash(at)gmail.com> wrote:
>
> Many people will instead copy the workgroup structure on
> every node, and connect to this local workgroup location.
> Maintaining the workgroup is a matter of copying the
> "updated" workgroup on every node (although I would recommend
> the disconnect-reconnect steps in such a pipeline).

With the speed enhancements in 5.1 the only reason I can see for copying
workgroups around is for speed where really large networks are concerned.
Form small to medium setups copying workgroups around is just a hassle most
people could do without.


---
Unsubscribe? Mail Majordomo(at)Softimage.COM with the following text in body:
unsubscribe xsi



---
Unsubscribe? Mail Majordomo(at)Softimage.COM with the following text in body:
unsubscribe xsi



--- Unsubscribe? Mail Majordomo(at)Softimage.COM with the following text in body: unsubscribe xsi


Search the XSI List archives here or use the advanced search form to search across mailing lists. Searching help is available.
This site supposedly brought to you by Benjamin Grosser and the Imaging Technology Group.