Volumetric rendering study

Raymarching clouds

A study of volumetric sampling, self-shadowing and light absorption in Unreal Engine 5.

My contribution
Volume shader research & implementation
Platform / environment
Unreal Engine 5
Tools & techniques
Unreal Engine 5HLSLMaterial FunctionsRaymarchingDistance Fields
Raymarching clouds — A study of volumetric sampling, self-shadowing and light absorption in Unreal Engine 5.

The project at a glance

The challenge

Understand how a sampled density field becomes a lit volume, with self-shadowing, shadows from the scene and control over sampling artifacts.

The approach

Break the effect down into a primary ray that accumulates density, secondary rays towards the light, soft shadows using the engine's Distance Fields, spatiotemporal dithering and per-channel absorption.

Result & scope

A didactic shader with stages that can be inspected separately. The main conclusion: volumetric raymarching is very expensive even on modern hardware.

Context and objective

The work of Mike Turitzin, who shares spectacular raymarching rendering demos on social media, gave me the idea of replicating the technique to render clouds. I am currently developing a raymarching engine from scratch; this small project was a way to scratch that itch and understand the fundamentals of the technique inside an engine I already knew, Unreal Engine 5.

The project is not aimed at performance or production reuse: it is a dissection of the process to properly understand each piece.

What raymarching is

In conventional rendering, the GPU rasterizes triangles: surfaces with no interior. A cloud has no surface: it is a volume of varying density that scatters and absorbs light throughout its interior.

Raymarching solves this by advancing a ray for each pixel in discrete steps through the volume and accumulating what it finds at each step. This is the core of RayStep, the function executed at each step of the primary ray:

float cursample = PseudoVolumeTexture(VolTex, VolTexSampler, saturate(pos),
                                      rmParams.XYFrames, rmParams.NumFrames).r;
if (cursample > 0.001)
{
    float shadowdist = ComputeLighting(rmParams, VolTex, VolTexSampler, pos);  // self-shadowing
    float dfshadow   = ComputeDFShadows(rmParams, pos);                         // scene shadows

    // Beer-Lambert law applied to the current sample
    cursample = 1 - exp(-cursample * rmParams.Density);

    // Accumulate the light that reaches this point and is scattered towards the camera
    lightEnergy   += exp(-shadowdist * rmParams.ShadowDensity) * cursample
                   * transmittance * rmParams.LightColor * dfshadow;
    transmittance *= 1 - cursample;
}

The attenuation follows the Beer-Lambert law: light passing through a medium decays exponentially with the density accumulated along its path. transmittance stores how much background light still passes through the volume, and lightEnergy, how much scattered light reaches the camera.

Primary ray and density sampling

The cloud is a 3D Perlin noise density field inside a box. Instead of evaluating the noise at each sample, I read it from a pseudo volume texture: a 2D texture that stores the slices of the volume in a grid of frames, which Unreal can sample as if it were a 3D texture. For each pixel, the primary ray enters the box, advances through it and accumulates the samples. The kernel, implemented as a Material Function with a Custom node, calls RayStep at each step and finishes with a partial step so it does not overshoot the exit of the box:

float3 lightEnergy = 0;
float transmittance = 1;

for (int i = 0; i < rmParams.MaxSteps; i++)
{
    // density and lighting at the current position of the view ray
    RayStep(rmParams, VolTex, VolTexSampler, CurPos, i, lightEnergy, transmittance, DebugOutput);
    CurPos += -rmParams.LocCamVec;
}

// partial final step up to the exit of the box
CurPos += rmParams.LocCamVec * (1 - rmParams.FinalStepSize);
RayStep(rmParams, VolTex, VolTexSampler, CurPos, rmParams.MaxSteps, lightEnergy, transmittance, DebugOutput);

return float4(lightEnergy, transmittance);

The number of steps is the parameter that most affects the result. With fewer steps sampling is cheaper, but detail is lost, banding appears and, in extreme cases, the shape of the cloud is no longer visible. I chose the values by tweaking the variables and visually comparing quality and cost.

Self-shadowing

A believable cloud gets darker in the areas that receive less light. To compute this, a second ray towards the directional light is cast from each point evaluated by the primary ray. That ray accumulates the density it finds along its path: the more density it passes through, the more occluded the point is.

This is the most expensive part of the algorithm, because it multiplies the number of samples: each step of the primary ray casts its own march. Two early exits avoid work that would not change the result:

  • The calculation stops when leaving the box of the volume.
  • It also stops when the accumulated density exceeds a threshold beyond which the point is already considered fully in shadow.
Cloud with self-shadowing

Cloud generated with 3D Perlin noise showing self-shadowing.

Both exits are visible in ComputeLighting, the function that marches towards the light:

float ComputeLighting(RaymarchParameters rmParams, Texture2D VolTex, SamplerState VolTexSampler, float3 CurPos)
{
    float3 lpos = CurPos;
    float shadowdist = 0;

    for (int s = 0; s < rmParams.ShadowSteps; s++)
    {
        lpos += rmParams.LightVector;
        // density sample at the current position of the light ray
        float lsample = PseudoVolumeTexture(VolTex, VolTexSampler, saturate(lpos),
                                            rmParams.XYFrames, rmParams.NumFrames).r;

        // components equal 1 if lpos is outside the box (local coordinates)
        float3 shadowboxtest = floor(0.5 + abs(0.5 - lpos));
        float exitshadowbox = shadowboxtest.x + shadowboxtest.y + shadowboxtest.z;

        if (shadowdist > rmParams.ShadowThreshold || exitshadowbox >= 1)
            break;

        shadowdist += lsample;   // accumulated density = occlusion
    }

    return shadowdist;
}

Scene shadows with Distance Fields

Self-shadowing is not enough: the cloud must also receive shadows from objects in the scene. For this I relied on the Distance Fields that Unreal already generates for each mesh, which make it possible to query the distance to the nearest object at any point.

From the evaluated point, the march advances towards the light, but instead of using an infinitely thin ray, it uses a cone that widens as it moves away. If an object enters the cone, it occludes part of the light; the farther away the occluder is, the wider the cone is at that point and the softer the penumbra (soft shadows), just as with real shadows.

Since Distance Fields are queried in world coordinates, the function converts positions to World Space at the start.

Cloud lit only by the light coming through a window

Only the light coming through the window lights the cloud.

float ComputeDFShadows(RaymarchParameters rmParams, float3 CurPos)
{
    // box position -> world space, relative to the camera
    float3 dfpos = 2 * (CurPos - 0.5) * GetPrimitiveData(rmParams.MaterialParameters).LocalObjectBoundsMax;
    dfpos = LWCToFloat(TransformLocalPositionToWorld(rmParams.MaterialParameters, dfpos)) - rmParams.CameraPosWS;

    float dftracedist = 1;
    float dfshadow = 1;
    float curdist = 0;
    float DistanceAlongTrace = 0;

    for (int d = 0; d < rmParams.DFSSteps; d++)
    {
        DistanceAlongTrace += curdist;
        curdist = GetDistanceToNearestSurfaceGlobal(dfpos);

        // virtual sphere that grows with distance: wider penumbra
        float SphereSize = DistanceAlongTrace * rmParams.LightTangent;

        // 0 = fully shadowed, 1 = fully lit
        // the closer the occluder is to the cone axis, the harder the shadow
        dfshadow = min(saturate(curdist / SphereSize), dfshadow);

        dfpos.xyz += rmParams.LightVectorWS * dftracedist * curdist;
        dftracedist *= 1.0001;
    }

    return dfshadow;
}

Ambient light

The directional light is not the only source: the sky lights the cloud from all directions. Computing this with rays in every direction would be prohibitive, so RayStep approximates it with three density samples upwards from the current point. The more density there is above, the less sky light the point receives:

shadowdist = 0;
shadowdist += PseudoVolumeTexture(..., saturate(pos + float3(0, 0, 0.25)), ...).r;
shadowdist += PseudoVolumeTexture(..., saturate(pos + float3(0, 0, 0.50)), ...).r;
shadowdist += PseudoVolumeTexture(..., saturate(pos + float3(0, 0, 0.75)), ...).r;

lightEnergy += exp(-shadowdist * rmParams.AmbientDensity) * cursample
             * rmParams.SkyColor * transmittance;

It is a very cheap approximation, but it is enough to make the base of the cloud darker and bluer than its top.

Reducing banding

As with any raymarching, reducing the number of steps causes banding: the process is discrete while the volume is continuous. All neighboring pixels sample at the same depths, so the error is coherent and forms visible steps.

To mitigate this, I slightly offset each sampling position along the camera vector, with a pattern that varies per pixel and per frame (spatiotemporal dithering). This jitter breaks the coherence of the quantization errors: the step turns into high-frequency noise, which the eye perceives as a smoother transition and which the engine’s TAA or TSR averages across frames.

The spatial component comes from a 4 × 4 Bayer matrix, and the temporal one shifts it every frame:

static const float BayerMatrix4x4[16] = {
     0.0f,  8.0f,  2.0f, 10.0f,
    12.0f,  4.0f, 14.0f,  6.0f,
     3.0f, 11.0f,  1.0f,  9.0f,
    15.0f,  7.0f, 13.0f,  5.0f
};

float get_dither_pattern_value(float2 seedA, float seedB)
{
    int x = int(seedA.x) % 4;
    int y = int(seedA.y) % 4;

    float base_dither_threshold = BayerMatrix4x4[y * 4 + x] / 16.0f;   // spatial pattern: 4x4 Bayer matrix
    float temporal_offset = frac(seedB * 0.38196601125f);              // temporal offset that changes every frame

    return frac(base_dither_threshold + temporal_offset);
}

Per-channel light absorption

In the calculation of the accumulated light energy, the absorption coefficient (rmParams.ShadowDensity) is not a scalar but a three-component vector. By default it has the same value in all three channels, but the shader allows a different value to be assigned to each one, so that the volume absorbs each wavelength differently:

lightEnergy += exp(-shadowdist * rmParams.ShadowDensity) * cursample * transmittance * ...;
//                               ^ float3: one absorption coefficient per channel

A channel with low absorption passes through the volume easily and tints the deep areas of the cloud; one with high absorption vanishes quickly. It gives the art team very direct control over how the volume transmits light.

Comparison of two absorption vectors

Left: absorption vector (0, 0.7, 1). Right: absorption vector (1, 0.1, 0).

What I learned

The main lesson was about cost: rendering volumes with raymarching is very expensive even on modern hardware. The number of samples grows with the resolution, the steps of the primary ray and the steps of each light ray, and every sample means one more texture read.

The techniques that make volumetric clouds viable in today’s games (rendering at a lower resolution, reprojecting results from previous frames, skipping empty space or approximating lighting with fewer samples) target precisely those multiplications, and understanding where the cost comes from was the best starting point for studying them in my own engine.

← All projectsGet in touch ↗