Multi-Tile UV Mapping/Displacement Part 4

By Brian Freisinger

June 9, 2012

(This tutorial is designed around Maya and Mudbox. Several of the concepts here can be transposed into other 3d software packages that have UV grid control)

This is a continuation of the multi tile UV mapping tutorial.

Click on the images to enlarge for more detail.

To understand how to layout the ‘tiles’ for maya and mudbox please refer to Multi-Tile UV Mapping Part 1

Here we will cover both how to use displacement with multi-tile UV mapping, as well as an alternate method of assigning multi-tile color.

Why 32 bit?

(if you already use 32 bit and don’t want a history lesson you can skip to the ‘how’ part)

First off, we’ll discuss 32 bit displacement vs 8 or 16 bit.

I highly recommend using 32 bit .exr format for anything you’ve done custom sculpting on in Mudbox or Zbrush.

8 and 16 bit have a variety of problems, so does .tif format.

The main problem with anything less than 32 bit is it only goes in a single normal direction. I.E. it’s only 16 (or 8) bit in or out.

What’s this mean? If you sculpt a landscape and have hills and valleys, and export a 16 bit, it’s either going to have only the hills or the valleys. To get both you’d need two 16 bit maps, one for the positive normal values (the hills) and one for the negative (the valleys)

16 in/16 out/water level

32 bit will handle both at the same time, so we only have a single file to work with.

Another pitfall to 16 bit or less is the need to adjust the map to display correctly. The user may need to shift the displacement +/- to get it to render 1:1 as expected from the sculpt.

32 bit knows about this internally based off the scale of the object, so it’s simply assign and render to get the same levels of displacement one would expect to see based off your sculpt.

IMPORTANT:

Because 32 bit does recognize the scale of the object you started with, change the scale AFTER you’ve sculpted and extracted displacement maps can have unexpected and mostly undesirable effects on your displacement render.

Scaling the object up will reduce the displacement amount to almost non existent, and scaling an object down will result in it looking ‘exploded’ or blown up.

So before painting a 32 bit displacement map it is very important to ESTABLISH THE SCALE OF YOUR OBJECT. If you paint on an object that is 5 cm across only to find out it needs to be 5 meters in the scene will cause some problems for you.

There are some workarounds using Nuke to change the scale, and it is possible to scale up and object in mudbox and then re-extract if a mistake is made, but it’s better to start with a properly scaled object.

Last thing. Mudbox saves an LZW compression tif. Bluntly this is horrible. Maya won’t read it, so you need to take it into Photoshop (which clamps your 32bit map and ruins the high/low details) or into Nuke to save it out in a non-compressed format. The EXR format saves properly, is lighter than tif and retains far more fidelity to the original sculpt.

In my personal opinion the whole .tif format is horrible. I use .png for color and .exr for displacement.

Ok, we’ll now move on to the how.

Extracting the displacement map.

multi tile layout

Using the same cube and six tile UV layout from part 1 we’ll paint a quick displacement map in Mudbox.

Two quick protips here:

“bleed”

One it’s good practice to ‘bleed’ your UV map at least 5% away from the edge of the 0,1 grid tile.

The reason here is mudbox might return some errors in the edge of the displacement if it doesn’t have some room for bleed calculations. This also applies to multiple shells in a UV tile. If you have several UV maps separated out in a single tile you should have space between each shell to get a good extraction.

Two, as discussed before double check your scale before exporting an object to mudbox.

My cube is 5 cm in XYZ for this tutorial.

subdivision on cube

I am also going to pre subdivide the cube now so it keeps it’s ‘cubeness’ instead of turning into a sphere on subdivision.

Major work of art. VES award here I come.

I did a quick displacement sculpt in Mudbox, nothing to fancy just enough to get the point across.

The point of this is to get the sculpt detail and the highs and lows to show up the same in our render.

level 0 issue

Another catch is mudbox has a nasty habit of effecting the base model when you’re sculpting.

So if we look back at our level 0 model we can see the high and low peaks I sculpted in are in the ‘zero’ level model.

I don’t know if this is a bug or a feature, but it’s a problem. You have two solutions here.

One is to export this model and use it as your new base mesh in Maya.

I do this often.

renamed sculpt

But in the event your model is so far down the pipeline this isn’t possible then I suggest the following.

Select your sculpted object and rename it to something like ‘highrezExtract’ or similar.

Original and Sculpt in scene

Next in the same scene import the original pre sculpted object.

You should now have two objects in your scene. One you sculpted and the original one you exported from Maya.

The yellow object is the selected ‘high resolution sculpt, the other is the one I started with by doing a Maya OBJ export.

“extracting from two models”

Now we  need to extract the displacement maps.

For mudbox that’s: Maps -> Extract Texture Maps

Obviously we’re doing displacement.

But because we have two objects we’re going to do a ‘difference’ between the two. So when we select our models we’ll want the original in “Target” and the high resolution in “Source”

This is why renaming them is handy. To know which goes where without confusion. Extraction can take a long time, so minimising mistakes is handy.

We also have to use the “Ray Casting” method because we’re doing a compare/difference between the two.

Closest to Target

One thing of note here:

I want to extract a 32 bit map with both in and out, so I need to make sure “closest to lowres mesh’ (i.e. closest to target) is selected. This will look both inside and outside the mesh.

There are several other tricks and settings in mudbox for getting a good displacement map out, but they can be found in tutorals all over the web, so I’ll leave that to you.

One little trick though is figuring out your search distance. I know my cube was 5cm so I want to make sure my search distance goes out beyond that since I sculpted “out” quite a bit.

I eventually made the search distance 1.0 so the high peak on the top wouldn’t clip.

file name and EXR type

Another quick tip:

I named my file cubemaps_01 not cubemaps_v01

A lot of people like to put a v for ‘version’ behind their file iterations. While not a bad practice it can come back to haunt you with Mudbox. Mudbox uses v and u to siginify which UV tile it belongs to (see previous tutorial)

So if I had named this cubemap_v01 instead I’d get a file outputs that said:

cubemap_v01_u1_v1.exr, cubemap_v01_u2_v1.exr, etc

Not only is this confusing on a quick glance when you need to assign your maps to the shader, it will also confuse mudbox if you need to reload your maps back onto a mesh in mudbox (Sculpt using map). Mudbox won’t be able to figure out what map belongs to what tile. So I strongly suggest not using the ‘v’ or version letter for your displacement maps.

Also use the OpenEXR 32 bit floating Point RGBA file type, not the Black and White.

I can’t speak for other programs, but Maya has a problem reading the Black and White.

4k maps

My maps are now extracted (I did 4k maps for detail)

As we’ve seen in the previous tutorials, I have six maps with coordinates exported (one for each tile)

Now we’ll need to create a shader to render these in Maya.

I’m using Vray, but this technique works for Renderman, Mental Ray or almost any other render engine on the market.

Creating the multi tile displacement shader

In my previous tutorial for color maps I used the layered texture shader. I’ve actually moved away from this because, well it sucks. I’m currently using the “plusMinusAverage” node for both my color and displacement.

The main reason here, is the layered texture shader in addition to being rather buggy, also does not respect negative values.

That means anything you paint going ‘IN’ won’t show up or worse show up as ‘clipped’ when you render. That’s a huge problem.

+/- Average

So the solution is to use the +/- average node.

That node will come in handly later, but lets start by setting up our shader.

I’m using Vray here so I’ll create a Vray shader. But if you’re using another render engine the principle is the same.

Create displacement node

I need to set up/create a displacement node that’s hooked into my shader.

for Vray I need to go to the SG (or shading group) and create a file node.

What this is doing (for Vray) is creating a vray native displacement node.

Again, if you’re using Arnold, Renderman, MentalRay, etc, hook your displacment node up.

displacement node

Once I’ve created it I actually delete the file node itself. I just need the displacement node to hook into later.

Attach luminance to displacement.

Next we’ll create a luminance node.

(NOTE: If you’re using this technique  of  plus/minus Average node instead of a layered shader for color maps DO NOT use the luminance node or it will change all your color maps to black and white)

This is the big trick to getting Maya to recognize negative values in a multi tile displacement set up. Without this node we will have our displacement all going “out” but nothing going in. So best case scenario it won’t look exactly like the sculpt, worst case it’ll clip and you’ll have smooth artifacts in the render. So save yourself pain and make a luminance node.

So if we go back to the displacement node we made, we’ll be replacing the file node we deleted with the luminance node. The easiest way to do this is just middle mouse drag the luminance node into the Displacement field of the displacement node.

If you use the connection editor it’s the ‘outvalue’ of the luminance node to the ‘displacement’ of the displacement node.

Attach +/- Average to Luminance

Now we’re going to use the +/- Average node we looked at earlier.

So create a “+/- Average” node and then middle mouse drag it to the “Value” field in the “Luminance“ node.

In the connection editor it’s the “3DOutput” of the +/- Average node connected to the “Value” of the Luminance node.

Create six “input 3d fields”

Now we’ll need to create slots in our +/- Average that will connect with our multiple displacement files.

This tutorial has 6 files, one for each side of our cube, so we’ll need to create six “slots”

So, select the +/- Average node you’ve created and go to the attribute editor.

You’ll see a field called Input 3D, and the option button to “Add New Item” if you click on this 6 times, you’ll now have six fields ready to be connected.

Create file node

We’ll now need six file nodes to hook up to the Input 3D field

There are a few things we’ll need to do with the file nodes next. A lot of these were covered in the previous multi tile color mapping tutorial, but we’ll go over them again.

File Node Attributes

Open the Attribute editor of your first file node.

FIrst I’ll load my first displacement map in which is cubemaps_01_u1_v1.exr

Next I’ll make my Default Color a solid black. If you fail to do this with either color maps or displacement maps you’ll get some rather undesirable effects. Because the node we’re using is additive we don’t want any other color introduced. I’ll spare you the long story, but just make sure you’re set to solid black. If you render looks weird, double check you did this.

Next we’ll go over the the place2dTexture node which is attached to the file node you made.

In here we’ll have a couple things to change.

Because this is our first map the “Translate Frame” values are left alone.

Mudbox numbers it’s first tile u1_v1, but in Maya it’s u0_v0

No, I have no idea why they do this.

Wrap U and Wrap V need to be unchecked. We have multi tiles, so we want one per tile at this point.

Just so we’re clear, I’ll load up my next file which is cubemaps_01_u2_v1.exr

next map

So you can see I loaded in cubemaps_01_u2_v1.exr

I made sure the default color was black (you have to do this for every node)

And in the place2dTexture node I’ve turned off the Wrap U and Wrap V again.

The Translate frame field is now set to 1 and 0

I’ve shifted it over 1 tile in u space, and none in v space.

u1_v1 was 0 and 0

u2_v1 is 1 and 0

A tile such as u5_v3 would be a translate value of?

4 and 2

It’s easy to loose track of this stuff when you have a lot of displacement tiles to keep some notes and double check your work often.

So do this for the last 4 file nodes. The should number up to 5 and 0 in Translate Frame

all nodes created.

Your hypershade should now look like this

We’ll now need to hook up the file nodes we created to the +/- average node.

Remember the six slots we made in the +/- Average node Output 3D field?

The file nodes will get attached to those.

Connection Editor

Open the connection editor (Windows->General Editors->Connection Editor)

Select the first file node and click “Reload Left” on the connection editor.

Select the +/- Average node and click “Reload Right” on the connection editor.

Now select “outColor” in the left column.

Expand the Input 3D fields by clicking the + sign and you’ll see the six fields we created earlier.

Select the first field “input3D[0]”

We’ve now established the connection between the file and the +/- Average node.

We’ll see the line appear in the hypershade showing the connection.

Now it’s a simple matter of doing this for the next 5 file nodes.

Just click on the next file node in the hypershade and click “Reload Left” in the connection editor again. Hook the next file nodes “outColor” to the next free field, “input3D[1]

Etc etc.

The connection order to the +/- Average node isn’t important.

The order of how they’ll read in render has already been established by the Translate Frame field in the place 2dTexture node.

displacement shader network.

When everything is finally hooked up your hypershade graph should look similar to this.

Make sure your shader is assigned to your object, also make sure you’ve flagged whatever subdivision surface attribute your render engine has for the object otherwise you won’t see very much.

Now we’ll render the scene.

Multi Tile Displacement Render

Success. It looks identical to the sculpt.

Basic troubleshooting for renders that don’t work.

Do I have the right files hooked up?

Did my displacement extractions actually work properly? (view them in Nuke or Photoshop to check)

Do I have my tiles and default color set correctly?

Did I remember to hook up a Luminance node.

Is it all piping into the correct displacement node.

And that’s it. Enjoy your multi tiling displacement.

 Brian Freisinger

October 18, 2011

Update: June 10, 2012 – To skip directly to how to set up for displacement jump ahead to part 4 of this tutorial series.

Also I’m no longer using the layered texture shader method and instead using the +/- Average node method as seen in Part 4. I’ll be updating this tutorial to reflect that in the future.

At the end of Part 2 you should have ended up with an object (Cube Primitive) laid out with multi-tile UV maps, and six texture maps you extracted from Mudbox.

There are now two ways you can assign your textures in a shader.

Again this applies to both texture and displacement mapping, but for this part of the tutorial we’ll just be looking at simple textures.

Let’s start by reloading the original cube scene you should have saved and then exported to Mudbox. If you forgot, just import your cube .OBJ you used in Part 2.

Simple Setup:

The first way is the down and dirty face by face assignment. I rarely use this unless I’m in a hurry or just want to run a couple tests without setting up a shader network.

It’s pretty simple to set up.

If we’ve got six tiles, and six textures, we simply create six shaders.

Figure 19. Shader per Face

I’m using Vray here and creating VrayMTL shaders, but this will work the same with Lamberts, Blinns or anything else.

You’ll also notice I named them with a u#_v# so I can keep not only my surface assignments straight, I can also quickly identify which texture goes with which shader.

Figure 20. Select each face and assign proper shader.

Now you simply grab each UV tile in order and assign the Shader to it.

This can be pretty tedious, but it’s simple and fast for diagnosing texture quality, or if you just want to check a couple of displacement maps on the fly without setting up a more complex network.

Figure 22. The correct texture tile for the correct shader.

Once that’s done, it’s pretty straight forward texture assignment per shader.

Figure 23. Make sure there is no Transparency.

There is one ‘gotcha’ when using .png files generated from Mudbox.

If you open the file in PhotoShop you’ll see it’s on ‘Layer 0’ with no background. Maya thinks that the empty background is a full alpha channel. So when it’s assigned to a shader, Maya tries to be helpful by adding a transparency or opacity channel.

So be aware, you either have to break the connection or lock the channel before adding the texture. I generally lock my opacity channels if I’m using this particular technique.

Both of these in Figure 23 have transparency added. The left is a generic Blinn, the right is a VrayMTL.

Double check this if you have render errors.

Figure 24. Render using the simple assignment technique.

If you wanted to add spec maps, reflection maps, bump, normal, or displacement maps, it’s the same method,  just the proper channels.

While this is the most simple and straight forward of setups, obviously it’s problematic for production because if you want to change Shader/Material attributes you have to make the same change to every shader, everytime. That’s why I only use this as a diagnostic tool, and not for standard production.

Master Shader:

The second and probably best method is to use a ‘layered texture’ node connected to a master shader: Blinn, Lambert, or Vray MTL etc.

The reason here is obvious, you have all your tiled textures under one node for surface property control. It takes a little more time to set up, but it’s worth it.

If you’re not familiar with the layered texture node it’s found under “other textures” in the HyperShade.

Using it can be a little tricky at times, but once it’s set up it can be invaluable.

Figure 25. layeredTexture node

Create a shader of your choice. For this one I’m going to use a lambert and name it “cubeMaster”

Create a render node under the “color” field, and assign the layered texture.

Figure 26. layeredTexture Attributes

Once it’s assigned should see the layered texture attributes.

Figure 27. Creating a new layer in the node.

Now select to the right of of the green swatch in the layered texture attribute edtor and create another node.

Just to make things simple, create four more for a total of six layers.

Six tiles, six layers.

Figure 28. Add your tile specific texture and make sure Blend is set to "Add"

Once that’s done we’ll assign our first tile to the first layer.

Take special node of the white boarder around what I have selected. It’s easy to mistakenly make changes to the wrong layer. Double check you see that white boarder that signifies “selected” before you make any changes or you might be doing it to the wrong layer.

You’ll also note in Figure 28, I highlighted the “Blend Mode”

It defaults with “none”, for a multi-tile texture set to work we need to set it to “add”

In the color field I’m assigning my first tile which is CubePaint_Tutorial_u1_v1.png

Figure 29. Set Default Color to solid black.

Because we’re using an “Add” Blend we’ll also need to set our default color to a solid black or it will contribute to the color.

If you forget to make this change it will start washing out your texture color until there is enough contribution to make it a solid white. It’s a very important step in the process to assure your painted textures and your rendered textures are 1:1

Figure 30. Uncheck wrap U and wrap V. Take note of the Translate Frame fields.

Next we’ll have to move down to the texture’s place2dTexture node and make some changes.

I have two areas highlighted in Figure 30.

The first is the translate frame. For our first tile we’ll do nothing, but this is important to take note of. It’s a bit counter intuitive to the process we’ve been using to count tiles, but the two fields of 0 and 0 basically mean u1_v1, or as it sees it zero translation in U or V. Basically it’s our first tile.

Second is the wrap u and wrap v fields. They’re on by default so your textures will continue repeating in each tile to infinity. Because we have textures for each tile painted we want to turn this off or the process will fail.

And that’s it, the first tile is done.

Figure 31. Keep Track of your layers.

Going back to our layered texture node, we now see tile one is blue, signifying it’s got a connection. You’ll notice the white outline around layer 2 which indicates I have it selected and am now ready to make it an “add” blend and assign the texture.

Figure 32. Translate the next texture/tile over by a full unit.

I go through the same process, making sure my default color is black, but this time in the place2dTexture node I make one change.

As I pointed out in Figure 30, I left the translate frame fields at the default 0,0

For my second tile, which I saved as CubePaint_Tutorial_u2_v1.png, I need to change the field to translate 1 in UV x, and 0 in UV v

This basically equals our u2_v1 tag.

So the next layer would have a Translate Frame of 2 for your texture u3_v1 and so on.

When a lot of files are flying around it’s easy to make the mistake of punching in the wrong numbers for your textures, so double check that if your render looks wrong.

So now it’s just a matter of going through the whole process till you have a full assignment of your tiles to the shader network.

Figure 33. Final Hypershade and Render.

In the end your Hypershade should look like this, and as you can see I did a quick render to make sure it was working. I’m using a VrayMTL in this one, but as I stated it works in any of the standard Lambert, Blinn, Phong, etc shaders.

From this point on I usually copy the network and start connecting it to the other channels I’m planning on using; Spec, Reflection, Bump, Displacement, or in the event of using a SSS shader, I’ll plug it into the Sub-surface color, Scatter color, etc.

That’s pretty much it.

Part 4, Multitile displacement mapping

Update: June 10, 2012 – To skip directly to how to set up for displacement jump ahead to part 4 of this tutorial series.

Also I’m no longer using the layered texture shader method and instead using the +/- Average node method as seen in Part 4. I’ll be updating this tutorial to reflect that in the future.

 Tags: CG, Vray, multiple UV, multiTile, multi-tile, UV mapping