RE: Lots of very high res textures

["kim aldis" <xsi(at)kim-aldis.co.uk>]

Consider this:-

 

 

 

So, we cast a set of ray samples through a single image pixel into an oblique, textured plane. In the instance above you’d (measuring by eye) need roughly 80x80 ray cast samples to get (roughly 40 textured pixels within the raycast cone, cf Cyquist sampling frequency needs to be double that of sampled item).

 

This is just one situation where increasing min/max sampling or texture resolution won’t help much. Because mip-mapping pre-samples the texture into decreasing sizes it should help. Eliptical filtering can also help but it can be expensive.

 

Which may or may not be what Harry is talking about, I didn’t read that far. (Sorry Harry J )

 

> -----Original Message-----

> From: owner-xsi(at)Softimage.COM [mailto:owner-xsi(at)Softimage.COM] On

> Behalf Of Harry Bardak

> Sent: 04 January 2008 12:44

> To: xsi(at)Softimage.COM

> Subject: RE: Lots of very high res textures

>

>

> I may add these about texture resolution that may help you. The problem

> you describe are common and are generate by this leitmotiv : I will got

> more details in my texture if I am using higher resolution.  Well that

> the common mistake and it 's not related to Mental Ray it's generic to

> any render system. You will need to resize down your texture to get all

> the details.

> Ok that sound weird, paradoxal ( it s a scandal monsieur !!!! , ......

> please choose the word you prefer )  so Why ? if you render let's in 2K

> a 8K texture ( with a lot of fine details ).

> For one pixel rendered you will get several texels to be render. So

> renderer ( Mr, PRman, Vray .... ) will average ( it's called filter )

> the texture to get theses texels to fit in the rendered pixel. The

> result is that you will loose all the details  of your texture. So the

> first reflex is to increase filtering sampling and AA value but it's

> very inefficient and your are not guaranteed to retrieve all the

> details.

> So having 8K textures are pretty useless until your render in 4K.

> But that doesn't mean you should paint your texture in lower

> resolution. Ask your texture artist to paint it in 8K.

> As Holger mentioned reduce the source and keep it with a good filter (

> sinc or lanczos ) the details will be kept and will be anti aliased.

>

> I see this question coming : Yes ok but if I convert my texture in a

> pyramidal .map file I should already get my reduced texture so Why

> should I need to scale down it ?

> Again this is a common mistake. 8K texture reduced in 2k with a sinc

> filter and then used as a 2K texture will reveal much more details than

> a 2K mipmaped version of a 8K texture.

> Why ? Again because of the filtering process. To choose the right

> mipmap resolution you need to look up the higher resolution and the

> lower resolution of you current mipmap version and filter them with the

> current one. If you don't do that you will end up with some transition

> problem if your object will move away of your camera.

>

> So reducing the texture will also reduce your space disk thus your

> network traffic thus decrease your render time.

> Hope it was clear, it's pretty unintuitive. But again I experimented

> theses stuff on The Golden Compass with PRman and I could stabilise

> render and memory usage and verify that I was not smoking cracks.

>

>

> More information about .map that I observed.

>

> - It looks like that MR load in memory only the texel visible of the

> texture. Since XSI keep the texture in memory when you do a render

> region. Your second render is cached.

> - If you add the new texture layer then you will have to rebuild your

> cache again. The first render may be slower but the next one should be

> faster.

> - Memory limits should be set everytime if you aim stability.

> - .map are network killer so have a local copy on your render node.

> - .map on a 32 bit system works on 64 bits system  not the opposite. I

> suspect the way the memory is managed in 64 bits is incompatible with

> 32 bits system.

> - tiled option increase speed but is incompatible with older version of

> MR.

>

> Others tips with texture is to use the 4 channels you have ( RGBA ) to

> store different type of data.

> eg.

> Spec map 1 in R

> Spec map 2 in G

> Reflection map in B

> Occlusion map in A

>

>

> Hope this help. It's not answering your question but may eradicate your

> problem.

>

>

>

>

> Harry Bardak TD / Compositor. Http://perso.wanadoo.fr/harry.bardak/ +33

> 6 76 63 35 54+44 781 661 4147

>

> ________________________________

> > Date: Wed, 2 Jan 2008 19:58:25 -0800

> > From: andy(at)thefront.com

> > To: xsi(at)Softimage.COM

> > Subject: Lots of very high res textures

> >

> > Okay, let me preface this by saying I know it's kind of ridiculous to

> try to render with a bunch of 8K 32-bit floating point textures.  I

> have an asset that I got from another vendor that came with multiple 8K

> maps for different parts of the shading (diffuse1, diffuse2, spec1,

> spec2, etc. -- like 8 maps per surface total).  And there are about 5

> different shaders on the asset.  So, all told, I'm looking at about

> 25GB of texture data (when uncompressed).  And, after converting the

> files to pyramidal .map files, I'm up to 53 GB.  Obviously, that's WAY

> overkill for the single frame of the asset at a sort of far distance

> that I have to render.  But it's not that common that I end up with an

> asset this heavy on textures, so I figured I'd have a go at getting it

> to actually render.

> >

> > I'm still experimenting, but so far, I've noticed some pretty

> interesting things.

> >

> > 1) I'm taking it for granted that I must use pyramidal .map files.  I

> didn't even bother trying this without doing that, but I'm pretty sure

> it's true.

> > 2) In conjunction with pyramidal .map files, elliptical filtering

> works at a reasonable speed for lookups on a single texture, and didn't

> seem terribly slower than disabling elliptical filtering (though I

> found that a little surprising).  So, most of my tests were done with

> elliptical filtering (20, 4, .3).

> > 3) For those of us like me who often lazily don't set a Mental Ray

> memory limit, you'll need to do that.  Otherwise, MR has an increased

> tendency to choke on memory allocation.

> > 4) If I draw a region, the render speed increases dramatically when I

> refresh.  For example, an experimental "first round" render time was

> about 1m31s, whereas refreshing the region took only 16s.  My

> hypothesis for this is that the hard disk's cache is simply doing its

> job.

> > 5) If I enable more than one texture per sample (for example, if I'm

> using both a spec map and a separate diffuse map), I experience

> dramatically increased render times.  For example, 3m12s vs 14m17s. My

> hypothesis for this is that the hard disk is having to seek back and

> forth between the two textures which live very far apart from each

> other on the disk (since they are 8K).

> >

> > Together, 4 and 5 suggest a possible optimization whereby one would

> benefit from simply rendering each texture individually to an empty

> buffer, then re-rendering the multi-textured surface samples

> (presumably pulling textures straight out of the cache the second time

> around).  Or, you could render in multiple passes, restricting each

> pass to a single texture (though this gets really really complicated,

> if not impossible, when you get to things like reflections, where

> you're casting multiple rays per sample).

> >

> > 6) For these renders, my CPU utilization is generally down to about

> 6%, for the obvious reason that all the time is being spent doing

> lookups on textures.  What I would really like is a way to manually

> "kill off" perceived texture resolution in my .map files, so that MR

> effectively ignores a specified number of the highest resolution layers

> of the pyramid, so that I could force myself back to renders with more

> reasonable CPU utilization.  While there's an obvious way to do this

> (down-res the texture and regenerate the .map), it's a lot of extra

> work that doesn't seem like it should really be necessary, since the

> resulting textures are already a subset of the existing ones.  I think

> there are ways to sort of do this, and I'm particularly interested in

> any information people have about that.  For example, I know that in

> the elliptical filtering settings, you can specify the maximum number

> of texels covered by the minor radius in the highest resolution

> pyramid.  My standard h!

>  as aways been 4, but for example, I was able to decrease the render

> above that took 3m12s down to 1m37s by taking this value down to 2.

> That's an improvement, but nowhere near what would be needed to get

> back to more reasonable render times.  I've already been over this a

> good while back, but it's pretty annoying that MR reads the texture

> filter bias (the multi-resolution texture blurring parameter on the

> clip) from the .map file (basically because they chose to implement on-

> disk memory mapping for the entire image structure, instead of just the

> data portion of it, as they should have).

> >

> > Anyway, I'm not holding my breath for any major revelations about how

> to optimize a scene with lots of gigantic textures (unless someone's

> got some?), but I just thought this was a good opportunity to push the

> envelope a bit, do a few tests and think about possible improvements

> for the future.  As storage gets more and more affordable, 8K textures

> and "taller" pyramidal maps are going to become more and more common.

> Not so much because you really need the higher resolution, but because

> you "could" need it on a close-up render of an asset, and it's often

> easier to just paint one high resolution texture and have it be dealt

> with dynamically.  I think that at some point, MR needs a re-write of

> the direct-from-disk texturing method.  A multi-channel format would be

> a good start, to optimize situation #5 above.  A new format should

> ideally also be able to completely emulate the behavior of lower

> resolution textures with as little performance loss as possible.  And

> on the imple!

>  mentation side, it should be optimized as much as possible for read-

> ahead.

> >

> > But honestly, I'm still kind of impressed that it's even stable (32-

> bit Windows).

> >

> > Now, I'll just shut up and down-rez all my textures to 1/4th

> resolution.

> >

> > - Andy

>

> _________________________________________________________________

> Express yourself instantly with MSN Messenger! Download today it's

> FREE!

> http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

> ---

> Unsubscribe? Mail Majordomo(at)Softimage.COM with the following text in

> body:

> unsubscribe xsi