Well, I don't think I have too much information. I do have something like
120 000 points that are used to calculate the collisions and I've created
something similar to a BSP tree to separate those points/boxes and be able
to extract only the points I need depending on the particles position but
where it's long is the process I do to each particle. If I have 1000
particles to process, it takes about a minute a frame which is reasonable
but I need to process like 50 000 particles and each particle doesn't
influence each other so that's why I'm looking to separate them in chunks of
1000 particles on 50 machines, each taking about a minute to process and
very few information has to be returned after the process.
Of course, where I'm not sure is how to pass those 120 000 points, the bsp
I've build (which has a bunch of pointers pointing to other pointers) and
that chunk of 1000 particles to another machine for processing and have that
machine pass me back those results. Of course, the bsp can be rebuilt since
it's already built quite rapidly and those 120 000 points exists in a scene
already so nothing is keeping me from reopening that scene, rebuilding the
bsp and just passing that chunk of 1000 particles, evaluate what I need and
pass back the result but again, I'm not sure how to call that from inside a
custom command.
The only way I see it for the moment is to write a temp file with the chunk
of particle information, call a script that'll open the scene, read the
file, do the process, write the result to another file and then read that
file when it's done to get the necessary information. But then how do I know
when to read the file? maybe use a master file where each computer goes in
and writes ok next to his thread once it's done and have the main computer
check that all processes are ok before reading the appropriate files...
that's quite ugly and I know there's a better way to do it.
I've never really been good at this network stuff! I'm more of a Math,
Trigonometry, Physics simulation kinda guy. I've followed all the classes
but kinda put it in the vault in the back of my mind once the classes where
finished and never had to use them till now so if anyone can help me
remember and figure out the best way to do this kind of stuff, I'd be
eternally grateful!!
Mathieu
-----Original Message-----
From: owner-xsi(at)Softimage.COM [mailto:owner-xsi(at)Softimage.COM]On Behalf
Of kim aldis
Sent: Tuesday, April 25, 2006 3:14 PM
To: XSI(at)Softimage.COM
Subject: RE: network distributed particle simulations?
Distributed iterative processing has been done. one of the very early
SIGGRAPH papers on particles was primarily about multi-processing particle
systems and involved allocating individual particles to processors. This
works pretty well for networked processors, I'd imagine, because it doesn't
require shared data, so long as any given processor handles the entire life
of the particle. But this wouldn't work so well for fluid sims.
But for your problem I think you're out of luck. The issue isn't about the
ability to multi-thread so much as the ability to chuck information about
other particles across a network fast enough to make it worthwhile doing.
Even if it were possible, multi-threading across multiple processors is
highly specialised.
> -----Original Message-----
> From: owner-xsi(at)Softimage.COM
> [mailto:owner-xsi(at)Softimage.COM] On Behalf Of Kris Rivel
> Sent: 25-April-2006 19:24
> To: XSI(at)Softimage.COM
> Subject: Re: network distributed particle simulations?
>
> An hour per frame....wow, you don't mess around do you :-) I
> didn't think distributing particles was possible. I know
> that the Next Limit guys are working on this I think but the
> obvious problem is that you need the previous frame to
> simulate the next frame, etc.
>
> Kris
>
> Mathieu Leclaire wrote:
> > OK, I'm still working on my particle simulations here... A particle
> > cloud with a scripted event that calls a custom command on
> every frame
> > to calculate a special collision event, where it passes information
> > from one cloud to another through a UserDataBlob when
> certain particles collide.
> > Anyway, the thing is, for the number of particles that I need, and
> > even though I believe I've optimized the simulation as best
> I can, it
> > takes roughly an hour a frame to calculate my simulations. At 200
> > frames, well, it takes more then a week of simulation using one
> > machine. We have a pretty big render farm here so if I could find a
> > way to use it to split the simulations over multiple
> machines, then we
> > can make the simulations in a more reasonable time frame. So my
> > question is this: how can I distribute my simulations over
> a network?
> > What would be the best way to approach the issue? I need to
> have xsi
> > evaluate the cloud normally, make the particles move depending on
> > forces and once the particles are positioned correctly, my scripted
> > event is called that calls the custom command. How can I
> separate the
> > process of that custom command over multiple machines so that one
> > machine calculates lets say the first 100 particles, and
> the next one
> > calculates another group of 100 particles and so on, until all the
> > particles have been evaluated and all the resulting data
> can be passed
> > to the UserDataBlob... I just want to make sure you guys understand
> > that what needs to be distributed is the evaluation in my custom
> > command that happens between each frames and that,
> depending on that
> > result, the particles will change and affect the following
> frame. So
> > before the next frame can be evaluated, the current network process
> > must be done before the next frame can start. Any
> suggestions on how
> > to do something like that? Anyone had to deal with
> something similar? Any help or suggestions would be greatly
> appreciated.
> >
> >
> > Mathieu Leclaire
> > R&D Programmer
> > Hybride Technologies
> >
> >
> >
> > ---
> > Unsubscribe? Mail Majordomo(at)Softimage.COM with the
> following text in body:
> > unsubscribe xsi
> >
> >
> >
>
> ---
> Unsubscribe? Mail Majordomo(at)Softimage.COM with the following
> text in body:
> unsubscribe xsi
>
---
Unsubscribe? Mail Majordomo(at)Softimage.COM with the following text in body:
unsubscribe xsi
---
Unsubscribe? Mail Majordomo(at)Softimage.COM with the following text in body:
unsubscribe xsi
