Render Graph Rendering using WGPU

Exploring render engine development using the Rust programming language

Goal & Motivation

This project and subsequent article have the goal to explore game engine development in the Rust eco-system, but since a general game engine can have quite a monstrous size and is made of many parts, I decided to focus on writing a renderer:

How can I built a modular and easy to extend, but still performant renderer in Rust?

My motivation for this project has multiple facets. For one, I want to learn and explore Rust, as it has several new concepts I am not familiar with:

• An imperative & functional programming paradigm v.s. OOP (object oriented programming)
• A memory ownership & borrow model v.s. garbage collection or unmanaged

Apart from these concepts, it is a steady upcoming and maturing cross-platform language.

Another reason for choosing this subject, is wanting to learn how game engines are constructed. Which parts make it a whole and how can they interact with one another. I am somewhat familiar with the concepts relating to engine design, but unfamiliar with the practical implementation of such concepts.

For specifically choosing the renderer, I see the renderer as the most vital part of a game engine. Games are a mostly visual medium where graphics are the first thing praised and looked at when being reviewed and highlighted in marketing.

Challenges

A project does not come without it’s challenges and mine isn’t an exception either. The main challenges I needed to solve were:

• How do I program in Rust?
• How can a project with multiple different libraries be setup using Cargo & Rust?
• What is a renderer?
• How do I use modern graphics API’s?
• How can WGPU be used to built a renderer?
• What is a render graph?
• How can I implement a render graph using WGPU?

Development Environment

Before I delve into the how’s and why’s of a renderer, I need to setup and explore the development environment I am going to need for my project. To start off, I will explore the programming language and the ecosystem that comes with it, after which I will need to figure out and explain how I structured my project.

Rust

What is Rust and why use it for a game engine?

Rust is a modern programming language that offers high performance, reliable concurrency, and memory safety. It’s designed to help developers build fast, efficient applications by preventing segfaults and ensuring thread safety without the need for a garbage collector ^[1].

Based on these inherent qualities, it makes the language a great candidate for game engine development. Besides these idiomatic qualities, the modern ecosystem & tooling offers many positives such as:

• Built-in language interoperability with support for FFI using the platform specific C-ABI's ^[2]. This means that if the Rust community isn’t able to provide any language-native library for a specific problem, or there’s a need to interact with an existing non-Rust codebase, this can easily be achieved and even automated in some (if not most) cases ^[3]!
• Built-in dependency management using its package manager, Cargo ^[4]. Since dependency management is provided directly through it’s built system, there is an ever growing community driven amount of libraries ready to use for whatever problem that needs to be tackled.
• Built-in documentation tooling powered by rustdoc ^[5]. It generates documentation for projects in the form of an HTML, CSS & JavaScript based website with built-in search, autohide and other navigation features.

Memory Management

To guarantee safety and reliability, the language makes use of a concept called memory ownership & borrowing. Unlike languages that use garbage collection or manual memory management (like C# or C/C++), Rust introduces a set of rules enforced at compile time to ensure memory safety, prevent data races, and eliminate many common bugs related to memory usage.

What is the memory management model of Rust, and how does it work?

Ownership & Moving

Each value has a single owner, a variable that is responsible for the value’s memory. When the owner goes out of scope, the value is automatically dropped, and its memory is freed. All in all are the ownership rules as follows ^[6]:

• Each value in Rust has an owner.
• There can only be one owner at a time.
• When the owner goes out of scope, the value will be dropped.

These rules eliminate the need for a garbage collector and prevent memory leaks.

fn main() {
    let mut x = 5;

    x = 10; // Allowed, x is mutable

    let y = x;

    y = 5; // Not allowed, y is immutable
}

Another concept tightly related, is moving. A move occurs when the ownership of a value is transferred from one variable to another. So when a value is moved, the original variable no longer has access to the value and thus it cannot be used to access or modify the value anymore. The three most typical situations when a move happens are when:

• A value is assigned to another variable: if you assign the value of one variable (s1 in the example below) to another (s2), the ownership of that value is moved to the new variable (s2). In the example, after assigning s2, s1 is not valid anymore.

let s1 = String::from("hello");

let s2 = s1;

• Passing a value to a function by value: if a variable (s) is passed to a function by value, the ownership of that value is moved to the function’s parameter (param_one). In the example, s is not valid anymore after the call of take_ownership(), as the value is moved into the function and out of its original scope.

fn take_ownership(param_one: String) {
    println!("{}", param_one);
}

let s = String::from("hello");

take_ownership(s);

• Returning a value from a function: when a function returns a value (some_string), the ownership of that value is moved to the variable (s) that catches the return value in the calling context. In the example below, we create and return the value of some_string “hello” in the give_ownership function. After the return, the variable some_string is invalid, as the value it contained has been moved out of the variable, and thus the current scope, into s.

fn give_ownership() -> String {
    let some_string = String::from("hello");

    some_string
}

let s = give_ownership();

Even though this system gives us safety and reliability, it isn’t always efficient, especially when moving huge chunks of data. Sometimes only temporary access to a value is needed in which case moving it doesn’t make much sense. It would get very convoluted real fast.

References & Borrowing

To overcome the shortcomings of ownership & moving, references & borrowing are the answer to these problems. Rust allows creating references to values without taking ownership of them. This is known as borrowing. A reference can be interpreted as a pointer, but unlike a pointer, it is guaranteed to point to a valid value of a particular type for the lifetime of that reference.

Therefor, borrowing enables multiple parts of your code to immutably or mutably read data without taking over ownership and thus without the risk of accidentally dropping that data.

fn main() {
    let mut s = String::from("hello"); // Allow data modification of this value

    let len = calculate_length(&s); // Reference the value without allowing modification

    append_world(&mut s); // Reference the value and allow value modification
}

fn calculate_length(s: &String) -> usize {
    s.len()
}

fn append_world(s: &mut String) {
    s.push_str(" world");
}

The concept of references is built on the following rules ^[6]:

• At any given time, you can have either one mutable reference, or, any number of immutable references. An example:

fn main() {
    let mut s = String::from("hello");

    {
        let r1 = &mut s; // Allowed, there are no mutable references yet
        // let r2 = &mut s; Not allowed, r1 is still in scope
    }
    
    let r3 = &mut s; // Allowed, r1 is out of scope

    append_world(r3);
}

fn append_world(s: &mut String) {
    s.push_str(" world");
}

• References must always be valid (based on their lifetime). In the following example we contain a reference in a struct, which means we need to explicitly state the reference’s and, therefor, the struct’s lifetime:

struct Highlight<'a> { // 'a is used to define the applicable lifetime
    part: &'a str,
}

fn main() {
    let text = String::from("Hello, world!");

    let first_word_end = text.find(',').unwrap_or(text.len());

    let highlight = Highlight {
        part: &text[0..first_word_end], // Since we create the reference within the scope
    };                                  // of main, the highlight value is not valid outside
}                                       // of it

Setup & Organisation

Now that I have explored the language, I need to understand how to setup and organise my project. To do this, I will first turn the focus to source code organisation, after which I will shine a light on Cargo, Rust’s package manager & build system.

Source Code Organisation

How is a Rust source tree organised?

With a big project such as a game engine, it is highly important to make sure the codebase stays organised. In Rust, code organisation revolves around two key concepts: crates and modules. These structures help manage and organise code logically, facilitating reusability, modularity, and maintainability.

Modules

The first layer of code organisation in the source tree are modules. A module can contain functions, structs, traits, impl blocks, and even other modules, allowing related functionality to be grouped together and control the visibility of code. It has a file system-based approach by default, meaning that modules are often defined in their own files or directories, although they can also be defined inline in a single file for smaller projects, etc ^[6].

Crates

A crate is the basic unit of compilation in Rust and it can be compiled into a binary executable or into libraries for other projects to use. The entire set of functionality provided by a crate can be made accessible to other crates by publishing it on crates.io, Rust’s package registry, or using a link to a git repository holding a crate ^[6].

Cargo

Which package manager and build system does Rust use?

When working with multi part projects, it is important to correctly manage dependencies and code base specificities. One of the best traits of Rust tooling, is the simplicity it provides for tasks which for most language ecosystems are complex or burdensome to deal with, namely dependency management and build system configurations.

Setup

The tool to use when setting up these tasks is Cargo. It is the default project management system of Rust and you can easily create new Cargo projects using the cargo new command in the terminal. This creates a folder which holds a Cargo.toml and Cargo.lock file, a src and target (output) folder and it initialises a new git repository from the get go ^{[4] [6]}.

Dependency Management

To work with dependencies, Cargo handles downloading, building, and integrating external crates into the project ^{[4] [6]}. Just like .NET with the NuGet Gallery, Rust takes a centralised dependency repository approach with crates.io, but it also allows for other sources, such as git repositories containing a crate to be used.

[package]
name = "example"
version = "0.1.0"
# Defines which language edition to use, which in turn decides the available features
edition = "2021"

[dependencies]
# Features can be used to set dependency configurations
wgpu = { version = "0.19", features = ["spirv", "glsl"] }
pollster = "0.3"
# Optionally a branch, tag or commit can be chosen when using a git dependency
test = crate_name = { git = "https://github.com/user/repo.git", branch = "branch_name" }

Build System

The build process itself can be customised using build.rs scripts, which are executed before the rest of the build process. These scripts can be used for tasks like generating code before compilation, building non-Rust code (like C libraries), or automatically generating bindings to C libraries ^[4].

use anyhow::*;
use fs_extra::copy_items;
use fs_extra::dir::CopyOptions;
use std::env;

fn main() -> Result<()> {
    // This tells Cargo to rerun this script if something in /res/ changes.
    println!("cargo:rerun-if-changed=res/*");

    let out_dir = env::var("OUT_DIR")?;
    let mut copy_options = CopyOptions::new();

    copy_options.overwrite = true;

    let mut paths_to_copy = Vec::new();

    paths_to_copy.push("res/");

    copy_items(&paths_to_copy, out_dir, &copy_options)?;

    Ok(())
}

Workspaces

Going back to project organisation, when working with projects consisting of multiple crates, Cargo provides workspace support. This allows you to build and manage several crates that are developed together, sharing the same Cargo.lock file and target directory to avoid compiling the same dependencies multiple times for different crates in the workspace ^{[4] [6]}.

[workspace]
# The dependency resolution algorithm for managing
# the compatibility and compilation of dependencies
resolver = "2"
# List of members in the workspace
members = [
    "crate_a",    
    "libraries/crate_b", 
]

# E.g. of a workspaces specific setting, a dependency override
[patch.crates-io]
# Overrides the 'some_dependency' crate for all workspace members
some_dependency = { git = "https://example.com/some_dependency.git", branch = "master" }

project_root/
├── Cargo.toml          # The config file as defined above
├── crate_a/
│   ├── Cargo.toml      # Per crate Cargo configurations
│   └── src/
│       └── main.rs     # Defines a binary crate
└── libraries/
    └── crate_b/
        ├── Cargo.toml  
        └── src/
            └── lib.rs  # Defines a library crate

Project Setup

How am I going to setup my project?

For a game engine project as the one I am starting, I want the project to be modular with multiple libraries. To achieve this I setup a Cargo workspaces project with the following Cargo.toml file:

[workspace]
resolver = "2"
members = [
    "redbush_renderer"
]

For now I am not going to need much, but as the project grows it will be easy to extend and configure as needed.

redbush_engine/
├── Cargo.toml
├── Cargo.lock
├── target/
└── redbush_renderer/
    ├── Cargo.toml
    └── src/
        └── lib.rs

Renderer

For this article I decided to focus specifically on developing a renderer. However, before designing one, I first have to understand the concept of rendering and graphics programming, and how to do this in Rust.

Rendering

What is rendering in the context of game development?

In the context of computer graphics, specifically game development, rendering is the process of drawing the game’s visuals onscreen. It takes the game’s assets, such as textures, models, and lighting information, and interprets them into the final image that the player sees. The final image can be photorealistic or highly stylised based on the game artist’s intent.

In most games, this image creation process occurs in real-time, continuously as the game runs, for which the device it is running on needs to perform a lot of the same calculations at the same time. CPU’s aren’t optimised for this, they are designed around performing sequential differentiated workloads, so, to speeds up this process, the GPU comes into play. It is designed around performing a lot of the same calculations in parallel.

GPU

How is a GPU better supplied for parallel processing?

The parallel processing architecture of the GPU includes hundreds or even thousands of smaller cores capable of running concurrent processing tasks. The GPU’s cores are organised into larger groups or “blocks,” enabling it to handle multiple operations on large sets of data simultaneously ^[7]. This is particularly useful in rendering tasks, where the same operation, like shading or texturing, needs to be applied to thousands or millions of pixels of an image.

Graphics Programming

How can the GPU be programmed against?

To define the rendering process, a GPU can be programmed against, just like the CPU. There are many ways to write GPU programs (also known as shader programs or shaders), and not just for rendering, but also for scientific calculations, machine learning, etc. But in the case of game development, APIs focused on rasterisation (image creation) are preferred. Currently there are three widely supported modern APIs to achieve this:

Since these APIs all have the same end goal, a lot of their concepts overlap with one another.

Pipelines

One of these concepts, is that only specific parts of the graphics pipeline are programmable, while other parts are built into the driver as highly optimised stages. With the above mentioned APIs, we can distinguish three types of pipelines available to the programmer, namely: the traditional vertex-fragment, task/mesh-fragment and the compute pipeline.

Traditional Pipeline

The oldest and most well-known pipeline of these three, is the traditional vertex-fragment pipeline and in the table below, the available stages per API are described. Pink stages are built-in and cannot be programmed with custom logic, orange is required to be set with custom logic, yellow is optional

Task/Mesh Pipeline

The task/mesh pipeline is the newest and lesser known of these three. It revamps the vertex stage and it’s related shaders, but keeps the fragment stage as is.

Compute Pipeline

The compute pipeline is specifically designed to handle compute tasks and is available in all three of the APIs. These tasks can also be non-graphical in nature and can range from physics simulations to post-processing effects in graphics, audio processing, or any general-purpose computation that can benefit from the massive parallel processing power of modern GPUs. It consists of one compute shader to which multiple resources can be bound for reading and writing ^[8]-[10].

Each of the pipelines detailed above represents a single sequence of operations designed to transform 3D data into 2D images. However, creating visually rich and complex scenes often requires more than a single trip through the associated pipeline stages. This is where the concept of graphics passes come into play.

Passes

In graphics programming, a pass can be thought of as a sequence of operations executed with a specific rendering goal in mind. These operations (e.g. changing the pipeline and thus state, or copying data from one place to another) are typically issued as a set of commands to the GPU, which are recorded in command lists or command buffers ^[11]-[13]. Passes are fundamental to complex rendering processes, allowing for the layered application of effects, efficient post-processing, and strategic optimisations that enhance performance and visual appeal. Organising the rendering code into discrete passes also promotes code maintainability and reusability.

Resource Management

Another more technical reason for implementing passes, is resource state management. As passes are a series of related command executions that utilise and produce specific data, such as calculating shadows, lighting, or reflections. Each pass is designed to complete a particular task and may involve reading from and writing to various resources like textures and data buffers ^[11]-[13].

Therefor, effective resource management within and across passes is crucial, as for example, a depth map generated during a shadow pass may be subsequently utilised in a lighting pass. Modern APIs (especially Vulkan and DirectX 12) are highly explicit in their resource management, which means the complexity is left to the developer, whereas older APIs left a lot of the resource state management for the driver to figure out.

Project API

What do I expect to be able to implement for my renderer?

For my renderer I want to make sure I can guarantee cross-platform development for the three biggest desktop platforms: Windows, macOS and Linux. This means it should at least support Vulkan, DirectX 12 and Metal.

When it comes to pipeline and shader support, I expect to integrate the compute and task/mesh pipelines fully, but for the vertex-fragment pipeline I can only guarantee support for the vertex and fragment shader stages. As for pitfalls, I am expecting synchronisation & resource state management to become a problem and time hog, so I will have to find a way around this in my design.

Graphics Programming in Rust

How can I program against the GPU in Rust?

Based on the project requirements and expectations I set for myself, I currently have two options:

1. Directly building an abstraction library upon Vulkan, DirectX 12 and Metal with a single entry point and shared interface
2. Find an already existing library built upon Vulkan, DirectX 12 and Metal with a single entry point and shared interface

To achieve the first option, I can for example use the following combination of Rust crates:

• ash: a highly performant, very lightweight wrapper of the Vulkan API.
• d3d12: a collection of thin abstractions over Direct3D 12.
• metal: unsafe Rust bindings for the Metal 3D Graphics API.

These three combined would allow for creating an unified abstraction layer, but for existing unified APIs, per the latter option, I currently have two possible choices:

• wgpu: a cross-platform, safe, pure-rust graphics API.
• bgfx-rs: a Rust wrapper for bgfx.

After looking into all of these, my choice has gone for WGPU, as it is Rust native with a Rust-idiomatic API. It has been used in quite a lot of different projects already and has a very active & supportive community.

Exploring WGPU

How can I use WGPU to write a renderer?

One of the benefits of an active community is the existence of a multitude of examples, tutorials and in-depth documentation as to how to use a library. To get a feel for WGPU (and Rust itself) I followed the well-known Learn Wgpu tutorial by Ben Hansen.

What I quickly noticed while following the tutorial and digging through the other sources, is the explicitness required when using the API. It isn’t nearly as extreme as Vulkan, but it is nonetheless very noticeable it inherits this quality from being built on top of modern explicit APIs.

For example setting up a pipeline looks like this:

let pipeline = device.create_render_pipeline(&RenderPipelineDescriptor {
            label: Some("Render Pipeline"),
            layout: Some(&pipeline_layout),
            // Define vertex pass
            vertex: VertexState {
                module: &vertex_shader,
                entry_point: "main",
                buffers: &[Vertex::desc(), InstanceData::descriptor()],
            },
            // Define fragment pass
            fragment: Some(FragmentState {
                module: &fragment_shader,
                entry_point: "main",
                targets: &[Some(ColorTargetState {
                    format: config.format,
                    blend: Some(BlendState::REPLACE),
                    write_mask: ColorWrites::ALL,
                })],
            }),
            // Define how to handle meshes/topolgy
            primitive: PrimitiveState {
                topology: PrimitiveTopology::TriangleList,
                strip_index_format: None,
                front_face: FrontFace::Ccw,
                cull_mode: None,
                polygon_mode: PolygonMode::Fill,
                unclipped_depth: false,
                conservative: false,
            },
            // Define what to use for depth stenciling
            depth_stencil: Some(DepthStencilState {
                format: TextureFormat::Depth32Float,
                depth_write_enabled: true,
                // LESS means pixels will be drawn front to back
                depth_compare: CompareFunction::Less,
                stencil: StencilState::default(),
                bias: DepthBiasState::default(),
            }),
            // MSAA
            multisample: MultisampleState {
                count: 1,
                mask: !0,
                alpha_to_coverage_enabled: false,
            },
            multiview: None,
        });

Everything needs to be pre-defined and requires extensive configuration. Another example is defining the passes:

let mut render_pass = encoder.begin_render_pass(&RenderPassDescriptor {
            label: Some("Clear Pass"),
            color_attachments: &[Some(RenderPassColorAttachment {
                view: &view,
                resolve_target: None,
                ops: Operations {
                    load: LoadOp::Clear(Color {
                        r: 0.1,
                        g: 0.2,
                        b: 0.3,
                        a: 1.0,
                    }),
                    store: StoreOp::Store,
                },
            })],
            depth_stencil_attachment: Some(RenderPassDepthStencilAttachment {
                view: &self.depth_texture.view,
                depth_ops: Some(Operations {
                    load: LoadOp::Clear(1.0),
                    store: StoreOp::Store,
                }),
                stencil_ops: None,
            }),
            timestamp_writes: None,
            occlusion_query_set: None,
        });

        render_pass.set_pipeline(&self.pipeline);
        render_pass.set_bind_group(0, &self.texture_bind_group, &[]);
        render_pass.set_bind_group(1, &self.camera_bind_group, &[]);
        render_pass.set_vertex_buffer(0, self.vertex_buffer.slice(..));
        render_pass.set_vertex_buffer(1, self.instance_buffer.slice(..));
        render_pass.set_index_buffer(self.index_buffer.slice(..), IndexFormat::Uint16);
        render_pass.draw_indexed(0..self.indices_length, 0, 0..self.instances.len() as _);
        // Release the mutable borrow of the render pass
        drop(render_pass);
        // Submit the clear pass
        self.queue.submit(once(encoder.finish()));

Everything used as a resource had to be defined before the program started like the pipeline mentioned above.

While finishing up the beginner part of the tutorial, I found myself writing a lot of boilerplate code which seemed to be quickly expanding, especially if I were going to continue down the path as shown by the tutorial. To better understand how other renderers overcome this problem, I looked into the renderer design of Bevy^[19]. It is a Rust native, popular game engine, using wgpu for its rendering. To manage its render logic, the Bevy team implemented a concept called a render graph^[20].

Render Graph

What is a render graph?

One of the first, but mostly the most famous, mention of a graph based renderer, was done by Yuriy O’Donnell at the 2017 GDC. At EA they ran into the same problem of the Frostbite’s renderer becoming a huge monster with the explicitness required by modern APIs. It caused for the renderer to be hard to maintain and extend, with teams having to fork/diverge the engine’s source code to be able to customise it. Let alone merging and integrating said diversions.

Frame Graph

To overcome these problems and abstract away the explicitness of state and resource management, they implemented something called a frame graph. It makes use of the immediate mode rendering paradigm to keep track of changes to the rendering process in real-time, but then compiles this information into the required retained mode structure as needed.

To be more specific, for every frame the system builds a high level representation of the render passes and resources (the API exposes a virtual object of said types), which is then processed and compiled into an executable render process. While compiling, unreferenced resources and passes are culled, resource lifetimes are calculated based on usage and lastly all GPU resources are allocated based on usage.

The lifetimes and allocation being kept to the exact time they are needed allows for a great number of optimisations techniques to be applied, such as memory aliasing (instead of destructing and creating resources, reuse the memory the last one occupied). After compilation the frame graph is executed and rebuild from the ground up for the next frame.

All in all, the goals of this system were to:

• Simplify resource management
• Simplify render pipeline configuration
• Simplify async compute and resource barriers
• Allow self-contained and efficient rendering modules
• Visualise and debug complex rendering pipelines

Evolution

The concept of a frame graph changed into the more encapsulating render graph over time, which in most cases changes the approach to what happens after the compilation phase. Since instead of completely recompiling the whole graph every frame, the system executes the original graph, caches it, and checks the next virtual graph against the cache. If nothing changed, it will reuse the already compiled render graph and execute it ^{[17] [18]}.

Since this new evolvement of the concept, most existing non-/commercial engines have implemented their own version of the concept:

Project Goals

What are the goals and non-goals for my render graph implementation?

Even though a big motivator for other implementations is the performance optimisations it allows, I want to specifically focus on a streamlined public interface first and on performance second. This translates to the goals of simplifying resource management & pipeline configuration, and creating self-contained & reusable rendering modules.

Simplify Resource Management

WGPU has an implicit resource management model, meaning a lot of the management regarding placing barriers and synchronisation is handled by the API itself. It is, however, still required to explicitly tell how and when to use resources. To simplify this process, the graph should take over the declaration of transient resource usage.

Simplify Render Module Configuration

WGPU has an explicit configuration model, which decreases the ease of use and expands the need for boilerplate code. To simplify this process, the graph should automate most of configuration of render modules based on the passes and resources used.

Self-contained & Reusable Rendering Modules

My last goal is to make sure all parts of the render graph will be self-contained and reusable. This means, for example, that a pipeline can be reused in multiple passe, and passes can be reused multiple times in the graph.

Design

How do I want my API to work?

For the design of my public API, I relied heavily on the Render Graph 101 article by Manon Oomen at Traversal Research on what they learned about designing a public render graph API. The design is easy understand and declarative, while still being able to be very expressive.

What I describe in this chapter is the best case scenario, which means that the design is still subject to change based on the implementation and the problems I run into.

Resources

When designing a render graph, two types of resources can be distinguished:

• Imported resources: resources that are used by the render graph, but are not created by it.

These resources are created by the application and are passed to the render graph. They can be used by multiple passes and are not destroyed by the render graph. Most of the time, these resources are loaded from disk and are used for the entire duration of the application. They will use the “standard” way of creating and destroying resources in WGPU:

let vertex_buffer = device.create_buffer_init(
    &BufferInitDescriptor {
        label: Some("vertex_buffer"),
        contents: CUBE_VERTICES,
        usage: BufferUsages::VERTEX,
    }
)

let v_buffer = render_graph.import(&vertex_buffer);

• Transient resources: resources that are created by the render graph and are only used by it.

These resources are created by the render graph and are only used by the passes that are part of the specific render graph. They are destroyed when the render graph is destroyed or when they are no longer needed during the execution. Initialisation should be done by the render graph itself:

let storage_texture = render_graph.create_resource(
    CreationInfo::Texture {
        "storage_texture",
        Resolution::FullRes,
        TextureFormat::R32g32Uint,
    }
);

Internally, the render graph will keep track of all resources and their usage with virtual handles. These handles will be used to reference the resources in the passes and pipelines, and will be resolved to the actual WGPU resources when the render graph is compiled using their (transient) resource caches.

Passes

Passes will be the main building blocks of the render graph. They will be used to define the rendering process and will be declared by setting which resources they will use and how they will use them. The render graph will keep track of the passes and their dependencies, and will compile them into a render process when the graph is executed.

The API will most likely look like this:

RasterPass::new("Draw pass", &mut render_graph)
    .render_target(&color_buffer_0, LoadOp::Clear, StoreOp::Store)
    .render_target(&color_buffer_1, LoadOp::Clear, StoreOp::Store)
    .render_target(&normal_buffer, LoadOp::Clear, StoreOp::Store)
    .depth_stencil(
        &depth_buffer,
        LoadOp::Clear,
        StoreOp::Store,
        DepthStencilMode::DepthStencilRenderTarget,
        LoadOp::Clear,
        StoreOp::Store,
    )
    .draw_indexed(
        &raster_pipeline,
        IndexBufferFormat::Uint32,
        &index_buffer,
        128,
        0,
        1,
    );

Pipelines

When creating pipelines, WGPU requires defining a pipeline layout, which is a collection of data layouts as provided by the bind groups used for resources. The render graph should automate this process by creating the pipeline layout based on the resources used by the passes.

The public API for the pipelines should look like this:

let pipeline = RasterPipeline::new("Raster pipeline", &mut shader_cache)
    .vertex_shader("vertex_shader")
    .fragment_shader("fragment_shader")
    .depth_stencil_state(
        TextureFormat::Depth32Float,
        true,
        CompareFunction::LessEqual,
        StencilState::default(),
        DepthBiasState::default(),
    )
    .primitive_state(PrimitiveState::default())
    .multisample_state(MultisampleState::default())
    .color_state(
        TextureFormat::Rgba8Unorm,
        BlendState::default(),
        ColorWrite::ALL,
    );

Since the render graph will build these pipelines when the graph is compiled, they should be cached and reused when possible. This will prevent the render graph from creating the same pipeline multiple times, which will improve performance, skipping initialisation and compilation time.

Graph

The render graph itself will be the main structure that will keep track of the resources, passes, and pipelines. It is responsible for compiling the render process and executing it, but should be decoupled from the actual renderer. This should allow for custom renderers and multiple render graphs to be used in the same application.

let render_graph = RenderGraph::Default();

// Add resources & passes to the render graph

render_graph.compile_and_execute();

Internally the render graph should check if the logic has changed since the last compilation, and if not, it will reuse the already compiled render process. This will prevent the render graph from recompiling the same render process multiple times.

Implementation

Since the implementation has become pretty big, I define the different parts per related subject heading.

Resources

Both resource types are cached in their respective caches based on their assigned lifetime and are assigned virtual handles, which are translated to the actual resource reference on execution. The translation is performed with a lookup for the virtual handle:

let texture = match descriptor.view.lifetime {
                    GraphResourceLifetime::Intermediate => {
                        Some(intermediate_resource_cache.get_texture(descriptor.view.into()))
                    }
                    GraphResourceLifetime::Transient => {
                        transient_resource_cache.get_texture(descriptor.view)
                    }

                    GraphResourceLifetime::Surface => None,
};

Intermediate

Currently intermediate resources are created and added to the resource cache when the program starts, which looks like this:

let cube_vertex_buffer = resource_cache.import_buffer(buffer::Buffer::new(
            device.create_buffer_init(&BufferInitDescriptor {
                label: Some("vertex_buffer"),
                contents: cast_slice(CUBE_VERTICES),
                usage: BufferUsages::VERTEX,
            }),
        ));

Transient

Transient resources are created during the rendering process and cached to the transient cache behind the scenes by the graph:

let depth_stencil_target =
            self.graph
                .create_resource(CreationInfo::Texture(TextureDescriptor {
                    label: None,
                    size: Extent3d {
                        width: surface_config.width,
                        height: surface_config.height,
                        depth_or_array_layers: 1,
                    },
                    mip_level_count: 1,
                    sample_count: 1,
                    dimension: TextureDimension::D2,
                    format: TextureFormat::Depth32Float,
                    usage: TextureUsages::RENDER_ATTACHMENT | TextureUsages::TEXTURE_BINDING,
                    view_formats: vec![],
                }));

Actual creation happens at graph compile time:

let mut transient_resource_cache = TransientResourceCache::default();

        for (index, resource) in self.resources.iter().enumerate() {
            if let GraphResource::Transient(creation_info) = resource {
                transient_resource_cache.create_resource(
                    render_context.clone(),
                    index,
                    creation_info,
                );
            }
        }

Passes

The render graph currently only supports raster passes, which output a render texture. A pass is defined as follows:

RasterPass::new(
            Some("cubes_pass"),
            &mut self.graph,
            Some(DepthStencilTargetDescriptor {
                view: depth_stencil_target,
                depth_load_operation: LoadOp::Clear,
                depth_store_operation: StoreOp::Store,
                depth_clear_color: Some(255),
                stencil_load_operation: LoadOp::Clear,
                stencil_store_operation: StoreOp::Discard,
                stencil_clear_color: None,
            }),
        )
        .render_target(RenderTargetDescriptor {
            view: render_target,
            resolve_target: None,
            load_operation: LoadOp::Clear,
            store_operation: StoreOp::Store,
            clear_color: Some(Color32::new_opaque(128, 128, 0)),
        })
        .draw_indexed(DrawIndexedParams {
            pipeline: self.pipeline.clone(),
            bind_groups: vec![camera_bind_group],
            vertex_buffers: vec![cube_vertex_buffer, cubes_buffer],
            index_buffer: (cube_index_buffer, IndexFormat::Uint16),
            indices: 0..CUBE_INDICES.len() as u32,
            base_vertex: 0,
            instances: 0..self.cubes.len() as u32,
        });

Behind the scenes the actual raster pass value is dropped and its underlying generic pass value is added to the graph:

fn drop(&mut self) {
        self.render_graph.add_pass(self.pass.take().unwrap());
    }

During compilation they are compiled and readied for execution:

let compiled_passes: Vec<CompiledPass> =
            self.passes.iter().map(|pass| pass.compile()).collect();

        self.compiled_graph = Some(CompiledRenderGraph {
            render_context,
            passes: compiled_passes,
            shader_cache,
            intermediate_resources: resource_cache,
            transient_resources: Arc::new(transient_resource_cache),
        });

        self.next_frame(new_hash);

To support caching, the passes are hashed to produce a value to be checked against. Passes are executed on a compiled graph during execution into command buffers, which are then submitted to the GPU:

pub fn run(&self, surface_view: Arc<TextureView>) {
        let command_buffers = self
            .passes
            .iter()
            .map(|pass| {
                if pass.label.is_some() {
                    println!("{}", pass.label.as_ref().unwrap());
                }

                pass.execute(
                    self.render_context.clone(),
                    self.shader_cache.clone(),
                    self.intermediate_resources.clone(),
                    self.transient_resources.clone(),
                    surface_view.clone(),
                )
            })
            .collect::<Vec<_>>();

        self.render_context.get_queue().submit(command_buffers);
    }

Pipelines

The passes are mostly unchanged from the WGPU approach in the current implementation, as I was not able to automate the bind group creation process. See the Exploring WGPU heading for an example.

Graph

The graph creation has been implemented like the design and makes use of caching by first checking if the current graph is identical to the last, using the pass collection hash:

let new_hash = self.hasher.hash_one(&self.passes);

if self.previous_passes_hash.is_some() && self.previous_passes_hash.unwrap() == new_hash {
            println!("Skipping compilation of Render Graph");

            self.next_frame(new_hash);

            return;
}

Example Scene

To test the current implementation I have defined a simple scene which shows 9 cubes with random colours:

To showcase the caching working, I added debug output to the console:

Compiling Render Graph
Frame 1: Running Render Graph
Skipping compilation of Render Graph
Frame 2: Running Render Graph

Discussion

The implementation is still in a premature state and is currently held down by several limitations which were mostly influenced by time constraints, but also lack of knowledge and experience.

Limitations & Further Improvements

Currently there are limitations which, with more time, could be further improved on and even solved. Some of them are goals which could not be achieved, while others were completely out of scope or suddenly showed up while implementing. A rough rundown of the current limitations are:

• A lack of built-in profiling, debugging & testing systems

With the current implementation it is possible to use external tools such as Renderman & the XCode Metal debugger, but these only focus on the rendering process on the GPU itself. There is no real insight into how the graph is structured and flows.

• Explicit bind group declarations & pipeline creation

Bind groups in this version are explicitly declared, which is one of the main problems I wanted to solve, but I was not able to work on this. It exposes unnecessary complexity and should be automated.

• No pass culling based on resource reads & writes

Naive pass culling is the only version which is currently built-in into the graph. In future versions an algorithm based on if the pass’s output is used in the final output should allow for advanced pass culling.

• No pass sorting based on resource reads, writes and versioning

There is no pass sorting implemented, which means that passes have to defined in the order they should be performed which increases logic complexity. This can be optimised by implementing a sorting algorithm which checks all the reads & writes, to and from, and the versioning of resources to decide the order.

• No compute pass support

During this iteration I focused on implementing the raster pass and not the compute passes, but the groundwork has been laid to ease the implementation off.

Resources

[1] Rust Team, “Rust Programming Language,” Rust-lang.org, 2018. https://www.rust-lang.org/

[2] Rust Team, “FFI – The Rustonomicon,” doc.rust-lang.org. https://doc.rust-lang.org/nomicon/ffi.html

[3] Rust Team, “bindgen,” GitHub, Apr. 19, 2022. https://github.com/rust-lang/rust-bindgen

[4] Rust Team, “Introduction – The Cargo Book,” doc.rust-lang.org. https://doc.rust-lang.org/cargo/index.html

[5] Rust Team, “What is rustdoc? – The rustdoc book,” doc.rust-lang.org. https://doc.rust-lang.org/rustdoc/what-is-rustdoc.html

[6] Rust Team, “The Rust Programming Language – The Rust Programming Language,” doc.rust-lang.org. https://doc.rust-lang.org/stable/book/

[7] N. Hagoort, “Exploring the GPU Architecture | VMware,” The Cloud Platform Tech Zone. https://core.vmware.com/resource/exploring-gpu-architecture#section1

[8] Khronos, “Pipelines,” Vulkan Documentation. https://docs.vulkan.org/spec/latest/chapters/pipelines.html

[9] Microsoft, “Pipelines and Shaders with Direct3D 12 – Win32 apps,” learn.microsoft.com, Dec. 30, 2021. https://learn.microsoft.com/en-us/windows/win32/direct3d12/pipelines-and-shaders-with-directx-12

[10] Apple, “Using a Render Pipeline to Render Primitives,” Apple Developer Documentation. https://developer.apple.com/documentation/metal/using_a_render_pipeline_to_render_primitives

[11] Microsoft, “Direct3D 12 render passes – Win32 apps,” learn.microsoft.com, Dec. 30, 2021. https://learn.microsoft.com/en-us/windows/win32/direct3d12/direct3d-12-render-passes

[12] Khronos, “Render Pass,” https://docs.vulkan.org/spec/latest/chapters/renderpass.html. https://docs.vulkan.org/spec/latest/chapters/renderpass.html (accessed Apr. 08, 2024).

[13] Apple, “Render Passes,” Apple Developer Documentation. https://developer.apple.com/documentation/metal/render_passes

[14] Microsoft, “Executing and Synchronizing Command Lists – Win32 apps,” learn.microsoft.com, Dec. 30, 2021. https://learn.microsoft.com/en-us/windows/win32/direct3d12/executing-and-synchronizing-command-lists#synchronizing-command-list-execution-using-command-queue-fences

[15] “Yet another blog explaining Vulkan synchronization – Maister’s Graphics Adventures,” Maister’s Graphics Adventures, Aug. 14, 2019. https://themaister.net/blog/2019/08/14/yet-another-blog-explaining-vulkan-synchronization/

[16] Apple, “Metal Best Practices Guide: Persistent Objects,” developer.apple.com. https://developer.apple.com/library/archive/documentation/3DDrawing/Conceptual/MTLBestPracticesGuide/PersistentObjects.html

[17] R. Loggini, “Render Graphs,” Riccardo Loggini, May 31, 2021. https://logins.github.io/graphics/2021/05/31/RenderGraphs.html

[18] A. Joshi, “Render graphs,” apoorvaj.io, Jul. 30, 2020. https://apoorvaj.io/render-graphs-1/

[19] Bevy Team, “Bevy – A data-driven game engine built in Rust,” bevyengine.org. https://bevyengine.org/

[20] Bevy Team, “Bevy 0.6,” bevyengine.org, Jan. 08, 2022. https://bevyengine.org/news/bevy-0-6/#render-graphs-and-sub-graphs