Parallax Occlusion Mapping (POM) is a relatively simple method of simulating surface detail on a flat surface without the need to create more geometry. The technique can be quite expensive if overused however it is sometimes more appropriate than alternate methods such as tessellation. The full source code can be found here.
The Pros
POM is an effective method of simulating geometry where none exists. Due to relying solely on texture maps and with a customisable amount of samples this method can yield great results whilst remaining relatively cheap compared to other alternatives. Parallax Occlusion Mapping can also be used to simulate extra geometry on complex objects such as character models and due to it being a mapping technique it can be used in combination with others such as normal mapping for another level of detail, hemisphere mapping and others.
The Cons
The effect is not perfect. You may see above as the cube rotates it fails to properly connect at the real edges of geometry. Whilst this would be no problem for objects such as flooring where edges are rarely seen or can be easily covered if used on a wall an additional cube object would need to be used to cover the artifacts on the edges. This method is also essentially a ray-tracer. As with all ray-tracing it can be quite expensive if not properly regulated and will not be suitable for low end hardware such as mobile devices.
The Code
As we are simulating geometry and not generating more nothing new needs to be done in the vertex shader (nor do we need a geometry shader) since our normal map code. POM does use the TBN matrix. There is a link to the source code for the vertex shader at the top of this page so that you know what you will need.
(Please note code comments are removed from these snippets)
#version 330
in vec3 tangentPos;
in vec2 texCoord;
in vec3 camPos;
in vec3 lightPos;
layout (location = 0) out vec4 FragColour;
uniform vec3 lightColour;
uniform sampler2D textureMap;
uniform sampler2D depthMap;
So POM attempts to simulate geometry using motion parallax, the phenomenon that objects that are closer will move faster, whereas objects that are further away will move slower as a camera pans from one side to another. To do this we are going to move around the texture coordinates of an object to make it look like as the camera moves around the closer fragments are also "moving" with the camera and the further away fragments are not / are moving slower. To obtain the depth for each fragment and therefore its displacement amount we need to use a depth map: an inverted height map.
The aim of the calculation will be to find which texture coordinate to use through the depth map.
vec2 POM(vec3 camDir, vec2 texCoord, vec3 normals)
{
float samples = mix(30, 5, abs(dot(normals, camDir)));
float sampleDist = 1.0 / samples;
float currentDepth = 0.0;
vec2 offsetTexCoord = 0.1 * (camDir.xy / (camDir.z * samples));
vec2 currentTexCoord = texCoord;
float depthFromMesh = texture2D(normalMap, currentTexCoord).a;
We put all the POM calculations into its own function just to keep everything else clean and readable. This function returns a vec2 which will be the displaced texture coordinates that we can simply use when reading textures.
The first value is the number of samples. This is the number of times the ray-tracer will check for a collision between the value of 0 (The real mesh surface) and 1.0 (The lowest point inside the mesh). We mix the results between 30 and 5 based on the dot product between positive Z and camDir. This is an optimisation as when the view vector is parallel with the surface normal the displacement amount will be 0 or nearly 0 therefore less samples we be needed. sampleDist calculates the distance between each sample check. For example if there are 10 samples the distance will be 0.1. We also initialise our current depth into the mesh at 0 (Mesh surface).
offsetTexCoord defines the direction and amount the view vector is going to travel into the mesh per sample. The way this method works is we extend the viewing vector into the mesh for amount sampleDist and keep going until we find a collision defined by the heightmap. Once this collision is found the resulting texture coordinate we're at is used. This is why more samples means more accuracy: We can only check if a collision has happened per sample. The larger the distance between samples, the more inaccurate the final texture coordinate will be.
0.1 is a multiplier to define the parallax strength. It is important as depth values 0.0 - 1.0 are world space coordinates, meaning a cube sized 1x1x1 will have parallax the depth of the entire cube.
We also initialise our currentTexCoord value to the real texture coordinate. This value will be the one we return.
Finally we sample the depth map. I have saved the depth map information into the alpha channel of the normal map however a seperate texture can be used. We sample using currentTexCoord to check the depth.
while(depthFromMesh > currentDepth)
{
currentDepth += sampleDist;
currentTexCoord -= offsetTexCoord;
depthFromMesh = texture2D(normalMap, currentTexCoord).a;
}
Now for the very small yet all important loop. The loop keeps running until our current depth equals or exceeds the depth defined by the current texture coordinate of the depth map. This will mean a collision has happened and is therefore where our new texture coordinate lies.
We first increase the current depth by the sample distance.
We then negate the current texture coordinate by the offset amount. This correctly extends the view vector down to where it would be if the mesh was at our current depth.
And finally we update depthFromMesh by checking the depth value of the current texture coordinate before the while loop then compares this value to the current depth.
This process can easily be visualised in the diagram below.
vec2 prevTexCoord = currentTexCoord + offsetTexCoord;
float afterDepth = depthFromMesh - currentDepth;
float beforeDepth = texture(normalMap, prevTexCoord).a - currentDepth + sampleDist;
float weight = afterDepth / (afterDepth - beforeDepth);
vec2 finalTexCoord = prevTexCoord * weight + currentTexCoord * (1.0 - weight);
return finalTexCoord;
}
One property of POM as opposed to alternative methods such as relief or steep parallax mapping is that it uses linear interpolation to obtain the final texture coordinate. By interpolating between each point we reduce the stair-stepping aliasing as seen below caused by too few samples, allowing us to use less samples with a clean result and thereby optimising greatly. The downside is that linear interpolation is, of course, only an approximation and will cause the most erroneous results at a sharp change in depth.
void main()
{
vec3 normals = normalize(uNormals);
vec3 camDir = normalize(camPos - position);
vec2 offsetTexCoord = POM(camDir, texCoord);
if (offsetTexCoord.x > 1.0 || offsetTexCoord.y > 1.0 || offsetTexCoord.x < 0.0 || offsetTexCoord.y < 0.0)
discard;
The problem we face now is that the object is still shaped as its original mesh and when viewing at extremely shallow angles the texture appears to repeat to infinity. While this may be desirable in some cases we will look at how to prevent this from happening to make the simple cube look like a high polygonal mesh.
Quite simply, before we use the new texture coordinates we check to see if they exceed a value of 1.0 or 0.0 on the x and y axis. If so, we discard these fragments completely, resulting in a deformed mesh that follows the height map. This is still not a perfect result, especially in the case of a rounded mesh, as the texture coordinate displacement is only limited to within the mesh itself. This means that the silhouette of the object will still seem rather flat. A fix for this would be to generate polygonal "fins" on the edges of objects to extend the displacement out to the sides. This would be useful for pillars or trees.
float selfOcclusion(vec3 lightDir, vec2 texCoord, float initialDepth)
{
float coefficient = 0.0;
float occludedSamples = 0.0;
float newCoefficient = 0.0;
float steps = 1.0;
float samples = mix(30, 5, abs(dot(vec3(0.0, 0.0, 1.0), lightDir)));
float sampleDist = initialDepth / samples;
vec2 offsetTexCoord = 0.1 * (lightDir.xy / (lightDir.z * samples));
float currentDepth = initialDepth - sampleDist;
vec2 currentTexCoord = texCoord + offsetTexCoord;
float depthFromMesh = texture(normalMap, currentTexCoord).a;
while(currentDepth > 0.0)
{
if(depthFromMesh < currentDepth)
{
occludedSamples += 1.0;
newCoefficient = (currentDepth - depthFromMesh) * (1.0 - steps / samples);
coefficient = max(coefficient, newCoefficient);
}
currentDepth -= sampleDist;
currentTexCoord += offsetTexCoord;
depthFromMesh = texture(normalMap, currentTexCoord).a;
steps += 1.0;
}
if(occludedSamples < 1.0)
coefficient = 1.0;
else
coefficient = 1.0 - coefficient;
return coefficient;
}
In combination with normal mapping POM looks great. Normal mapping defines the finer detail while displacing the light to create self-occlusion. Remember when I said normal mapping looked bad for large details such as bricks? Well with POM that shadowing now works a treat. Self-occlusion isn't the be-all and end-all though.
Our displaced texture coordinates may look convincingly three dimensional but since there is no change in geometry, self-shadowing does not occur. This leads to the lighting of the object looking incorrect in many scenarios as light is not properly occluded behind each bump and within every facet. We create a new function which will act as a multiplier for the entire lighting equation. We are doing almost the same as the texture displacement function here but instead we use the light vector instead of the camera vector. Starting at the fragment in question we go back up the light vector towards the light position checking if the vector collides with the depth map once more.
If it does then one of two things can happen: an if statement is triggered checking whether a fragment is occluded or not. if not the multiplier returns 1.0 (no change) otherwise it returns a constant set value, typically 0.3 or so. This results in hard shadows. While it does not look as good it is certainly a faster alternative. Or for soft shadows the if statement again returns 1.0 for no change or the multiplier decreases with every occluded step, darkening the fragment and simulating soft shadowing.
An example of self shadowing can be seen below.