Using Shaders For GPU Computation
Shaders are not limited to drawing pixels. They can also be used to solve mathematical problems on the GPU. This style of programming is often called GPGPU: general-purpose computing on graphics processing units.
The basic idea is simple:
- The CPU creates input buffers and uploads data to GPU memory.
- A shader runs many parallel invocations on the GPU.
- Each invocation computes part of the result.
- The GPU writes results into an output buffer.
- The CPU waits for the GPU work to finish and reads the output buffer back.
No image has to be presented to the screen.
Rendering Shaders vs Compute Shaders
Vertex and fragment shaders are part of a graphics pipeline. They are designed around drawing primitives such as triangles.
Compute shaders are different: they are designed for general parallel work. A compute shader is dispatched directly with a grid size, not by drawing geometry.
Graphics shaders can be forced into compute-like work by rendering into textures or buffers, but compute shaders are usually the cleaner tool when the goal is math rather than pixels.
Data Flow Between CPU And GPU
GPU memory is separate from normal CPU memory on many systems. Even on integrated GPUs, graphics APIs still make the ownership and synchronization explicit.
The normal flow is:
At a high level, the CPU needs to do five things:
- Create GPU buffers for the input and output data.
- Upload input values to the GPU.
- Bind those buffers so the shader can access them.
- Dispatch the shader with enough invocations to cover the problem.
- Synchronize and copy or map the output buffer back to CPU memory.
Storage Buffers
For mathematical workloads, data is commonly passed through storage buffers.
In GLSL, these are declared with buffer blocks. In Vulkan and modern OpenGL, they are often called SSBOs: shader storage buffer objects.
Example GLSL declarations:
layout(std430, binding = 0) readonly buffer MatrixA {
float a[];
};
layout(std430, binding = 1) readonly buffer MatrixB {
float b[];
};
layout(std430, binding = 2) writeonly buffer MatrixC {
float c[];
};
This means:
- Binding
0contains input matrixA. - Binding
1contains input matrixB. - Binding
2contains output matrixC. readonlytells the shader not to write to the input buffers.writeonlytells the shader not to read from the output buffer.std430defines a predictable memory layout for buffer data.
Matrix Multiplication Example
For matrix multiplication, the goal is:
C = A * B
If A is M x K and B is K x N, then C is M x N.
Each output element is independent:
C[row, col] = sum(A[row, i] * B[i, col]) for i from 0 to K - 1
That independence maps well to the GPU. One shader invocation can compute one output element of C.
Example Compute Shader
This GLSL compute shader computes one matrix element per invocation:
#version 450
layout(local_size_x = 16, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) readonly buffer MatrixA {
float a[];
};
layout(std430, binding = 1) readonly buffer MatrixB {
float b[];
};
layout(std430, binding = 2) writeonly buffer MatrixC {
float c[];
};
layout(push_constant) uniform Dimensions {
uint m;
uint k;
uint n;
} dims;
void main() {
uint col = gl_GlobalInvocationID.x;
uint row = gl_GlobalInvocationID.y;
if (row >= dims.m || col >= dims.n) {
return;
}
float sum = 0.0;
for (uint i = 0; i < dims.k; ++i) {
float av = a[row * dims.k + i];
float bv = b[i * dims.n + col];
sum += av * bv;
}
c[row * dims.n + col] = sum;
}
Important details:
local_size_xandlocal_size_ydefine the size of each workgroup.gl_GlobalInvocationID.xselects the output column.gl_GlobalInvocationID.yselects the output row.- The bounds check handles matrices whose sizes are not exact multiples of the workgroup size.
- Each invocation writes exactly one value in
C, avoiding write conflicts.
Dispatch Size
The CPU dispatches enough workgroups to cover the output matrix.
For a 16 x 16 local size:
groups_x = ceil(N / 16)
groups_y = ceil(M / 16)
groups_z = 1
The 16 is a practical default rather than a requirement. For the reasoning behind it, the hardware limits it must respect, and how different workgroup shapes trade off, see Compute Workgroup Sizing.
In C or C++ code, the calculation often looks like:
uint32_t groups_x = (n + 15) / 16;
uint32_t groups_y = (m + 15) / 16;
Then the API-specific dispatch call runs the shader:
// OpenGL style
glDispatchCompute(groups_x, groups_y, 1);
// Vulkan style
vkCmdDispatch(command_buffer, groups_x, groups_y, 1);
Passing Data To The GPU
The exact API calls differ between OpenGL and Vulkan, but the shape of the operation is the same.
In OpenGL, this usually means:
- Create buffers with
glGenBuffers. - Allocate and upload data with
glBufferDataorglNamedBufferData. - Bind buffers to shader storage bindings with
glBindBufferBase. - Set uniforms or push-like values for dimensions.
- Call
glDispatchCompute.
Conceptual OpenGL flow:
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, matrix_a_buffer);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 1, matrix_b_buffer);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 2, matrix_c_buffer);
glUseProgram(compute_program);
glDispatchCompute(groups_x, groups_y, 1);
In Vulkan, this usually means:
- Create
VkBufferobjects for inputs and outputs. - Allocate and bind
VkDeviceMemory. - Use a host-visible staging buffer or mapped memory to upload data.
- Bind buffers through descriptor sets.
- Bind a compute pipeline.
- Call
vkCmdDispatchinside a command buffer.
Conceptual Vulkan flow:
vkCmdBindPipeline(cmd, VK_PIPELINE_BIND_POINT_COMPUTE, compute_pipeline);
vkCmdBindDescriptorSets(cmd, VK_PIPELINE_BIND_POINT_COMPUTE, pipeline_layout,
0, 1, &descriptor_set, 0, nullptr);
vkCmdPushConstants(cmd, pipeline_layout, VK_SHADER_STAGE_COMPUTE_BIT,
0, sizeof(Dimensions), &dimensions);
vkCmdDispatch(cmd, groups_x, groups_y, 1);
Getting Data Back To The CPU
After dispatching the shader, the CPU cannot immediately assume the result is ready. GPU work is asynchronous.
The CPU must ensure two things:
- The shader writes are complete and visible to later copy/read operations.
- The GPU has finished copying or exposing the output data to CPU-visible memory.
In OpenGL, a simple path is:
glDispatchCompute(groups_x, groups_y, 1);
glMemoryBarrier(GL_SHADER_STORAGE_BARRIER_BIT);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, matrix_c_buffer);
void* data = glMapBufferRange(GL_SHADER_STORAGE_BUFFER, 0, output_size,
GL_MAP_READ_BIT);
// Copy data into CPU memory here.
glUnmapBuffer(GL_SHADER_STORAGE_BUFFER);
For real applications, persistent mapped buffers, sync objects, and double-buffering can avoid stalling the CPU every frame.
In Vulkan, a common path is:
- Insert a pipeline barrier from compute shader writes to transfer reads.
- Copy the device-local output buffer to a host-visible readback buffer with
vkCmdCopyBuffer. - Submit the command buffer.
- Wait on a fence.
- Map the readback buffer memory with
vkMapMemory. - Invalidate non-coherent memory if required.
- Read the result array on the CPU.
Why Synchronization Matters
Without synchronization, the CPU may read old or incomplete data. The GPU may also reorder work unless barriers tell it which memory operations must be visible before later operations run.
Synchronization is one of the biggest differences between CPU-only code and GPU programming. The shader code might be mathematically correct, but the result can still be wrong if memory visibility is not handled correctly.
Compute Without A Window
A compute workload does not need to present an image to the screen.
With Vulkan, you can create an instance, select a physical device, create a logical device with a compute queue, and run compute command buffers without creating a window or swapchain.
With OpenGL, compute shaders still require an OpenGL context. A context is often created through a windowing library, but it can be hidden or headless depending on the platform and driver.
Practical Notes
- GPUs are fastest when the same operation runs over large amounts of data.
- Small matrices may be slower on the GPU because upload, dispatch, synchronization, and readback overhead can dominate.
- Reading results back to the CPU is expensive compared with keeping data on the GPU.
- For best performance, keep intermediate data on the GPU and only read back final results.
- Matrix multiplication can be optimized with shared memory, tiling, and vectorized loads, but the simple one-output-element-per-invocation version is easier to understand.
Summary
To use shaders for mathematical computation:
- Put input data into GPU buffers.
- Bind those buffers to a compute shader.
- Dispatch enough shader invocations to cover the problem.
- Write results into an output buffer.
- Synchronize GPU work.
- Copy or map the output buffer so the CPU can read the result.
The same GPU execution model used for rendering can also solve parallel mathematical problems. The main difference is that compute shaders write to buffers or images instead of producing fragments for the screen.