Re: Lots of very high res textures

[Francois Lord <xsilist(at)francoislord.com>]

We've had the same problem recently on an asset with huge textures 
rendermapped from DarkTree.
After trying different things with the elliptical filtering settings and 
.map files, we ended up using a mix of 4K and 2K textures and native 
DarkTree shaders with a higher AA. It was still faster to render than 
the original 8K textures.

One thing that is not very useful about .maps files is that when you 
create them on 64bit machines, you can use them only on 64bit machines.

Andy Jones wrote:
> Okay, let me preface this by saying I know it's kind of ridiculous to 
> try to render with a bunch of 8K 32-bit floating point textures.  I 
> have an asset that I got from another vendor that came with multiple 
> 8K maps for different parts of the shading (diffuse1, diffuse2, spec1, 
> spec2, etc. -- like 8 maps per surface total).  And there are about 5 
> different shaders on the asset.  So, all told, I'm looking at about 
> 25GB of texture data (when uncompressed).  And, after converting the 
> files to pyramidal .map files, I'm up to 53 GB.  Obviously, that's WAY 
> overkill for the single frame of the asset at a sort of far distance 
> that I have to render.  But it's not that common that I end up with an 
> asset this heavy on textures, so I figured I'd have a go at getting it 
> to actually render.
>
> I'm still experimenting, but so far, I've noticed some pretty 
> interesting things.
>
> 1) I'm taking it for granted that I must use pyramidal .map files.  I 
> didn't even bother trying this without doing that, but I'm pretty sure 
> it's true.
> 2) In conjunction with pyramidal .map files, elliptical filtering 
> works at a reasonable speed for lookups on a single texture, and 
> didn't seem terribly slower than disabling elliptical filtering 
> (though I found that a little surprising).  So, most of my tests were 
> done with elliptical filtering (20, 4, .3).
> 3) For those of us like me who often lazily don't set a Mental Ray 
> memory limit, you'll need to do that.  Otherwise, MR has an increased 
> tendency to choke on memory allocation.
> 4) If I draw a region, the render speed increases dramatically when I 
> refresh.  For example, an experimental "first round" render time was 
> about 1m31s, whereas refreshing the region took only 16s.  My 
> hypothesis for this is that the hard disk's cache is simply doing its 
> job.
> 5) If I enable more than one texture per sample (for example, if I'm 
> using both a spec map and a separate diffuse map), I experience 
> dramatically increased render times.  For example, 3m12s vs 14m17s. My 
> hypothesis for this is that the hard disk is having to seek back and 
> forth between the two textures which live very far apart from each 
> other on the disk (since they are 8K).
>
> Together, 4 and 5 suggest a possible optimization whereby one would 
> benefit from simply rendering each texture individually to an empty 
> buffer, then re-rendering the multi-textured surface samples 
> (presumably pulling textures straight out of the cache the second time 
> around).  Or, you could render in multiple passes, restricting each 
> pass to a single texture (though this gets really really complicated, 
> if not impossible, when you get to things like reflections, where 
> you're casting multiple rays per sample).
>
> 6) For these renders, my CPU utilization is generally down to about 
> 6%, for the obvious reason that all the time is being spent doing 
> lookups on textures.  What I would really like is a way to manually 
> "kill off" perceived texture resolution in my .map files, so that MR 
> effectively ignores a specified number of the highest resolution 
> layers of the pyramid, so that I could force myself back to renders 
> with more reasonable CPU utilization.  While there's an obvious way to 
> do this (down-res the texture and regenerate the .map), it's a lot of 
> extra work that doesn't seem like it should really be necessary, since 
> the resulting textures are already a subset of the existing ones.  I 
> think there are ways to sort of do this, and I'm particularly 
> interested in any information people have about that.  For example, I 
> know that in the elliptical filtering settings, you can specify the 
> maximum number of texels covered by the minor radius in the highest 
> resolution pyramid.  My standard has aways been 4, but for example, I 
> was able to decrease the render above that took 3m12s down to 1m37s by 
> taking this value down to 2.  That's an improvement, but nowhere near 
> what would be needed to get back to more reasonable render times.  
> I've already been over this a good while back, but it's pretty 
> annoying that MR reads the texture filter bias (the multi-resolution 
> texture blurring parameter on the clip) from the .map file (basically 
> because they chose to implement on-disk memory mapping for the entire 
> image structure, instead of just the data portion of it, as they 
> should have).
>
> Anyway, I'm not holding my breath for any major revelations about how 
> to optimize a scene with lots of gigantic textures (unless someone's 
> got some?), but I just thought this was a good opportunity to push the 
> envelope a bit, do a few tests and think about possible improvements 
> for the future.  As storage gets more and more affordable, 8K textures 
> and "taller" pyramidal maps are going to become more and more common.  
> Not so much because you really need the higher resolution, but because 
> you "could" need it on a close-up render of an asset, and it's often 
> easier to just paint one high resolution texture and have it be dealt 
> with dynamically.  I think that at some point, MR needs a re-write of 
> the direct-from-disk texturing method.  A multi-channel format would 
> be a good start, to optimize situation #5 above.  A new format should 
> ideally also be able to completely emulate the behavior of lower 
> resolution textures with as little performance loss as possible.  And 
> on the implementation side, it should be optimized as much as possible 
> for read-ahead.
>
> But honestly, I'm still kind of impressed that it's even stable 
> (32-bit Windows).
>
> Now, I'll just shut up and down-rez all my textures to 1/4th resolution.
>
> - Andy
---
Unsubscribe? Mail Majordomo(at)Softimage.COM with the following text in body:
unsubscribe xsi