Interesting info, Andy - thanks for sharing. Best idea I came up with
was to get a 64bit machine with 128GB of ram and make a ramdisk to
keep the .map files on :D
Though I'd be interested if you do find a way to speed up the seek. It
sounds like every time it wants to access the image it may be going to
the start of the .map file to read/calculate some sort of index. If
this info was cached it may speed it up, but short of writing your
shader for sampling image files (or mental images implementing it) I
can't see a way of achieving that.
Cheers,
Alan.
On Jan 3, 2008 3:58 AM, Andy Jones <andy(at)thefront.com> wrote:
> Okay, let me preface this by saying I know it's kind of ridiculous to try to
> render with a bunch of 8K 32-bit floating point textures. I have an asset
> that I got from another vendor that came with multiple 8K maps for different
> parts of the shading (diffuse1, diffuse2, spec1, spec2, etc. -- like 8 maps
> per surface total). And there are about 5 different shaders on the asset.
> So, all told, I'm looking at about 25GB of texture data (when uncompressed).
> And, after converting the files to pyramidal .map files, I'm up to 53 GB.
> Obviously, that's WAY overkill for the single frame of the asset at a sort
> of far distance that I have to render. But it's not that common that I end
> up with an asset this heavy on textures, so I figured I'd have a go at
> getting it to actually render.
>
> I'm still experimenting, but so far, I've noticed some pretty interesting
> things.
>
> 1) I'm taking it for granted that I must use pyramidal .map files. I didn't
> even bother trying this without doing that, but I'm pretty sure it's true.
> 2) In conjunction with pyramidal .map files, elliptical filtering works at a
> reasonable speed for lookups on a single texture, and didn't seem terribly
> slower than disabling elliptical filtering (though I found that a little
> surprising). So, most of my tests were done with elliptical filtering (20,
> 4, .3).
> 3) For those of us like me who often lazily don't set a Mental Ray memory
> limit, you'll need to do that. Otherwise, MR has an increased tendency to
> choke on memory allocation.
> 4) If I draw a region, the render speed increases dramatically when I
> refresh. For example, an experimental "first round" render time was about
> 1m31s, whereas refreshing the region took only 16s. My hypothesis for this
> is that the hard disk's cache is simply doing its job.
> 5) If I enable more than one texture per sample (for example, if I'm using
> both a spec map and a separate diffuse map), I experience dramatically
> increased render times. For example, 3m12s vs 14m17s. My hypothesis for
> this is that the hard disk is having to seek back and forth between the two
> textures which live very far apart from each other on the disk (since they
> are 8K).
>
> Together, 4 and 5 suggest a possible optimization whereby one would benefit
> from simply rendering each texture individually to an empty buffer, then
> re-rendering the multi-textured surface samples (presumably pulling textures
> straight out of the cache the second time around). Or, you could render in
> multiple passes, restricting each pass to a single texture (though this gets
> really really complicated, if not impossible, when you get to things like
> reflections, where you're casting multiple rays per sample).
>
> 6) For these renders, my CPU utilization is generally down to about 6%, for
> the obvious reason that all the time is being spent doing lookups on
> textures. What I would really like is a way to manually "kill off"
> perceived texture resolution in my .map files, so that MR effectively
> ignores a specified number of the highest resolution layers of the pyramid,
> so that I could force myself back to renders with more reasonable CPU
> utilization. While there's an obvious way to do this (down-res the texture
> and regenerate the .map), it's a lot of extra work that doesn't seem like it
> should really be necessary, since the resulting textures are already a
> subset of the existing ones. I think there are ways to sort of do this, and
> I'm particularly interested in any information people have about that. For
> example, I know that in the elliptical filtering settings, you can specify
> the maximum number of texels covered by the minor radius in the highest
> resolution pyramid. My standard has aways been 4, but for example, I was
> able to decrease the render above that took 3m12s down to 1m37s by taking
> this value down to 2. That's an improvement, but nowhere near what would be
> needed to get back to more reasonable render times. I've already been over
> this a good while back, but it's pretty annoying that MR reads the texture
> filter bias (the multi-resolution texture blurring parameter on the clip)
> from the .map file (basically because they chose to implement on-disk memory
> mapping for the entire image structure, instead of just the data portion of
> it, as they should have).
>
> Anyway, I'm not holding my breath for any major revelations about how to
> optimize a scene with lots of gigantic textures (unless someone's got
> some?), but I just thought this was a good opportunity to push the envelope
> a bit, do a few tests and think about possible improvements for the future.
> As storage gets more and more affordable, 8K textures and "taller" pyramidal
> maps are going to become more and more common. Not so much because you
> really need the higher resolution, but because you "could" need it on a
> close-up render of an asset, and it's often easier to just paint one high
> resolution texture and have it be dealt with dynamically. I think that at
> some point, MR needs a re-write of the direct-from-disk texturing method. A
> multi-channel format would be a good start, to optimize situation #5 above.
> A new format should ideally also be able to completely emulate the behavior
> of lower resolution textures with as little performance loss as possible.
> And on the implementation side, it should be optimized as much as possible
> for read-ahead.
>
> But honestly, I'm still kind of impressed that it's even stable (32-bit
> Windows).
>
> Now, I'll just shut up and down-rez all my textures to 1/4th resolution.
>
> - Andy
>
---
Unsubscribe? Mail Majordomo(at)Softimage.COM with the following text in body:
unsubscribe xsi
