The Sobel filter is a method of feature detection. Not only does it find the outline to an object but also sharp edges within the mesh itself by looking for sudden changes in colour. Gradients can therefore get in the way but by adding a threshold to the resulting values certain sobel lines can be taken and added to the final result, giving a nice outline effect to use in combination with the Cel shading or a cross-hatching effect. The full source code can be found here.
(Please note code comments are removed from these snippets)
Shader "My Shaders/Sobel"
{
Properties
{
_MainTex ("Buffer", 2D) = "white" {}
_Threshold ("Outline Threshold", Range (0, 1)) = 0
_ScreenWidth ("Screen Width", float) = 0.0
_ScreenHeight ("Screen Height", float) = 0.0
}
SubShader
{
Pass
{
As usual we start with the setup stuff. First we define the name of the shader. In Unity this will be the path in the dropdown menu the user will have to take when searching for shaders. We set the name of the shader to Sobel, placing it in the My Shaders folder.
We the define the inputs of the shader, or the Properties. The underscore is an identifier for input variables. We grab the screens buffer and store it as a 2D texture with a default value of "white" (1.0, 1.0, 1.0, 1.0) if the buffer for whatever reason can't be found. We also take a threshold value which is a float ranging from 0 - 1. The threshold defines at what shade of grey we start to ignore the filter and use the buffers original colours instead. We also get the screens width and height from a script attached to the camera. I'll go into why we need this a little later on.
SubShader contains all of our shaders. It is named SubShader as it contains all of the shaders in one file; Vertex and fragment, and multiple passes if necessary.
CGPROGRAM
#pragma vertex vert_img
#pragma fragment frag
#include "UnityCG.cginc"
uniform sampler2D _MainTex;
uniform float _Threshold;
uniform float _ScreenWidth;
uniform float _ScreenHeight;
We tell the compiler to stop reading in ShaderLab and instead start compiling for CG, Nvidia's own shader language.
Pragma's are used to define the names of the vertex and fragment shader. As a vertex shader isn't needed for this post-process effect the standard built-in vertex shader is used.
Next we take the inputs of the shader and uniform them in to this specific pass (again, only one pass is needed here). 'uniform' doesn't actually have to be defined but for consistency and understanding of the code it is left in.
float4 frag(v2f_img input) : COLOR
{
float offsetX = 1.0 / _ScreenWidth;
float offsetY = 1.0 / _ScreenHeight;
float4 result = float4(0.0, 0.0, 0.0, 0.0);
float4 sumX = float4(0.0, 0.0, 0.0, 0.0);
float4 sumY = float4(0.0, 0.0, 0.0, 0.0);
float sum = 0.0;
float4 temp = float4(0.0, 0.0, 0.0, 0.0);
We start the fragment shader with yet more setting up. We obtain the offset amounts by dividing screen height and width from 1. In doing this, we find the distance between texture coordinates. As the Sobel filter finds sharp changes in colour we have to sample our current texture coordinate with its surrounding texture coordinates. Using these offset values we are able to check neighbouring pixels. This is where the importance of screen height and width come in. Without proper values here the offsets will be incorrect and therefore the sampling will be more incorrect or of a lower resolution than it should.
After this we initialise some values we'll be using later in the filter itself or as the resulting colour.
temp = tex2D(_MainTex, input.uv + float2(-offsetX, -offsetY));
sumX += temp * -3.0;
temp = tex2D(_MainTex, input.uv + float2(offsetX, -offsetY));
sumX += temp * 3.0;
temp = tex2D(_MainTex, input.uv + float2(-offsetX, 0.0));
sumX += temp * -10.0;
temp = tex2D(_MainTex, input.uv + float2(-offsetX, 0.0));
sumX += temp * 10.0;
temp = tex2D(_MainTex, input.uv + float2(-offsetX, offsetY));
sumX += temp * -3.0;
temp = tex2D(_MainTex, input.uv + float2(offsetX, offsetY));
sumX += temp * 3.0;
Now before we get into the filter itself and the code above an array of 9 float2's should be visualised. This array looks like this...
offset[0] = vec2(-offsetX, -offsetY);
offset[1] = vec2(0.0, -offsetY);
offset[2] = vec2(offsetX, -offsetY);
offset[3] = vec2(-offsetX, 0.0);
offset[4] = vec2(0.0, 0.0);
offset[5] = vec2(-offsetX, 0.0);
offset[6] = vec2(-offsetX, offsetY);
offset[7] = vec2(0.0, offsetY);
offset[8] = vec2(offsetX, offsetY);
...however won't directly be used in the code above. This array contains the offsets for all neighbouring texture coordinates and the original texture coordinate obtained by adding or subtracting our offset values. We will use certain neighbouring pixels for each half of the filter depending on whether we're sampling the X axis or Y axis.
So as you can see in the code above we store the colour value of our buffer texture into the temp value, offsetting the texture coordinates by the specific offsets. This value is then cumulatively added to the sum of the X axis contribution. Note how we multiply these results by either 3, 10, -3 or -10. This is a convolution matrix applied to check for the gradient. The most common convolution matrices used by the Sobel filter is either using 3 and 10 or 1 and 2 and yield different results.
temp = tex2D(_MainTex, input.uv + float2(-offsetX, -offsetY));
sumY += temp * 3.0;
temp = tex2D(_MainTex, input.uv + float2(0.0, -offsetY));
sumY += temp * 10.0;
temp = tex2D(_MainTex, input.uv + float2(offsetX, -offsetY));
sumY += temp * 3.0;
temp = tex2D(_MainTex, input.uv + float2(-offsetX, offsetY));
sumY += temp * -3.0;
temp = tex2D(_MainTex, input.uv + float2(0.0, offsetY));
sumY += temp * -10.0;
temp = tex2D(_MainTex, input.uv + float2(offsetX, offsetY));
sumY += temp * -3.0;
sum = sqrt((sumX * sumX) + (sumY * sumY));
The same is now done to the pixels related to the Y axis of the current pixel and values are cumulatively added to the sum of the Y axis of the Sobel filter, once again applying the convolution matrix. to their respective pixels.
Now that we have the final results of both the X axis and Y axis the two are combined to obtain the magnitude of the gradient. This returns a value between 0 and 1 where 1 is an extremely sharp gradient, or change in colours, and 0 is no gradient whatsoever, Returning this sum value alone outputs the result of the Sobel filter. Since sum is a single floating point value we can use it in all three colour channels to get the magnitudes of gradient in greyscale which is very nice, but not all that useful to us.
float4 textureMap = tex2D(_MainTex, input.uv);
//result = float4(sum, sum, sum, 1.0);
if(sum > _Threshold)
result = float4(0.0, 0.0, 0.0, 1.0);
else
result = float4(textureMap.rgb, 1.0);
return result;
}
ENDCG
}
}
}
So now we have the results of the sobel filter it's time to do something with them. Above I mentioned that this feature detection method can be useful for creating outlines and "inlines" of objects but for that to happen we need to extract only the sharpest of gradients only, and replace everything else with the original buffer colour. This is where the threshold value comes in.
Since the result of the Sobel filter is the magnitude of the gradient the whiter values are the important edges. So before setting our result we check to see is the resulting sum float is larger than the threshold. If so, we make the fragment black (Or whatever colour outline you want). If not, we use the original buffers texture instead. The threshold essentially removes all the smaller gradients that you can see the light causes across the planes in the images above. The larger the threshold, the bigger the griadient is ignored, resulting in thinner outlines.
This is a post process method of outline detection and as such the filter can only go by colours and not actual geometry. Because of this certain outlines may be lost. For example, the intense bright light removes outlining for part of the pedestal due to fragment colours not having enough gradient so be wary of this when using it in realism scenes with intense lighting.
