Easy Filters - Intro to GPU Pixel Shaders
In today’s article, we are going to explore the basics of low-level GPU computations used for rendering graphics. In order to do so, we are going to dive into the world of GPU Shaders and perform some image manipulations like contrast, brightness, blur, and pixellation. By the end of this article, you will be able to perform basic pixel operations using top graphics libraries like OpenGL and Metal (iOS, macOS).
If you want to perform real-time image filtering like in the above app preview you can download the iOS app
Enhance your image with just one tap! Simple social media sharing (instagram, facebook, snapchat). Easily edit, trim…
and continue reading to implement such effects by yourself!
Before we start with implementing GPU shaders, let’s start by asking ourselves - why do we need GPUs in the first place?
Regardless of the fact whether we are dealing with raster or vector graphics,
at the end of the day, our screens are just matrixes of pixels, where each pixel is defined by its RGB value.
For example, a FullHD screen contains 1920×1080=2073600 pixels.
So to have a smooth display, we need to find a way to efficiently compute over 2 millions of pixel values for every frame as fast as possible.
It’s possible to perform such computations on CPUs but it wouldn’t be optimal. CPUs are suited for heavy serial tasks but we need to compute many light tasks that can be computed in parallel and this where GPUs stand out.
What’s more, GPUs have built-in hardware acceleration for some linear algebra functions and can compute many of them at once - this is why GPUs are heavily leveraged in Graphics, Machine Learning and Blockchain worlds.
We need to compute many pixel values for each frame as fast as possible and now we know that in order to do it, we should use GPUs.
But how can we do it?
Let’s leverage GPU’s parallelism and create relatively small low-level programs (kernels) that can compute multiple pixel values simultaneously. Such programs all called Shaders.
The main idea behind the Gutenberg’s Press was to draw the whole page at once in a repetitive way.
This is almost exactly what we are doing in GPU shaders but instead of using types for drawing characters on paper, we are computing RGBA values for screen displays.
Pixel Shader is a program that is executed for every pixel of the drawable to calculate its RGBA value.
Usually, when learning a new technology/language we start with a Hello World program that outputs ‘Hello World’ to the console. It would be an overkill to start with such example in learning GPU Shaders so let’s start with a simple example that displays the red color on the screen.
For a given input vec2 fragCoord with coordinates, we output vec4 fragColor with corresponding RGBA value that we would like to show on the screen.
We define red color as
vec3(1.0, 0.0, 0.0);
which is nothing else as
RED=255, GREEN=0, BLUE=0
FYI: color values in shaders are normalized (0.0–1.0)
Since we want a red color for every pixel, we simply output our redColor vector and with an additional alpha channel of 1.0 to fragColor.
Ultimately, we are getting a red display. Pretty simple, huh?
Feel free to play with this basic example in your web browser!
Let’s proceed with something slightly more advanced and create a shader that presents a texture on the screen.
We need to calculate normalized coordinates.
For each pixel from the drawable, we are taking the corresponding pixel from the iChannel0 texture and assign its color to the originalColor variable.
Finally we are outputting the originalColor. Which results in simply presenting the iChannel0 texture on the screen.
Now let’s proceed with some image manipulation and perform some brightness adjustment to the input texture.
Let’s define our brightness adjustment value.
Then, we need to increment each RGB channel with the brightness value.
Finally, we need to output our new brightness adjusted color values and apply clamping because we don’t want to go over 1.0 and below 0.0.
In this example let’s play with contrast.
Let’s define our contrast adjustment value.
From every RGB value, we are subtracting 0.5 and then multiply the result by the contrast value. Finally, we are adding 0.5 to even out the initial subtraction.
Let’s move on to something more advanced - BoxBlur. The idea behind it is simple. For every pixel, we are going to calculate the average value of all surrounding pixels in a defined radius.
Let’s define our blurSize value.
Let’s calculate our range i.e how far should we look for every pixel in every direction (left, right, up, down)
Let’s define empty vec4 colors container.
The core of the BoxBlur algorithm. For every pixel, we are going to scan surrounding pixels within the defined range and sum their color values.
FYI: we are going to include source pixels as well
For blurSize of 25 we are going to check 12 pixels to the left, 12 pixels to the right, 12 pixels up and 12 pixels down. It will give us a sum of 25*25=625 values for every pixel.
Finally, we are calculating the final average color so we need to divide our vec4 colors by the blurSize squared (625 in our example).
Finally, let’s implement a pixellation effect.
Let’s define our size value.
We need to set our normalized square pixelSize.
We need to calculate the position to read the color from the input texture. To achieve the pixellation effect we can take every n value (which is defined by pixelSize) from the input texture and we can do it with the floor function.
In this tutorial, we’ve learned how to leverage GPUs for rendering graphics and simple image manipulations with OpenGL and Metal. However, we can achieve way more with GPU shaders and I encourage you to play with them yourself. You can start right away in your browser and feel free to use my kernels as starters!
Shadertoy WebGL Kernels
gsurma - Shadertoy BETA
Project’s GitHub page.
iOS metal camera with GPU shaders. Contribute to gsurma/metal_camera development by creating an account on GitHub.
Corresponding iOS app
Questions? Comments? Feel free to leave your feedback in the comments section or contact me directly at https://gsurma.github.io.
And don’t forget to 👏 if you enjoyed this article 🙂.