Real-Time VFX Systems for Luxury Brand Interactive Installation

Project Breakdown

This project’s client prefers the final output to remain private, but the R&D process and everything I built along the way felt worth sharing.

The goal was to create a roughly five-minute interactive experience where users traverse a themed 3D universe using their hands as motion guides, captured via an Orbbec Femto Bolt RGB-depth camera. Based on the choices made during that journey, the experience would reveal their “soul” in the form of a final point cloud effect that morphed into one of the client’s products.

This project pushed my understanding of rendering 3D effects through 2D texture arrays and helped me build critical intuition around Unreal’s Niagara 3D Grid interface systems. More broadly, it laid the groundwork for how I approach this class of problem in future projects.

Here is a edited cut of the experience walkthrough only showcasing the effects

I was tasked to create 3 real-time effects:

Boid “Cursor” System

Prior to starting this project I was already experimenting with and looking into niagaras grid 2d interface which is just unreals way of exposing compute shaders to the user. Very serendipitously one of the OG’s of the rt-vfx scene Ghislain Girardot released an extremely well documented video on Boids. I proposed this idea of then using a boids flock made up of gems as a “cursor” so that the users can see what selection/path they are making.

The key insight was that since we had a first-person camera mostly guided on rails, I could parent the Niagara system directly to the camera’s forward vector. This gave me a stable grid of coordinates for the effect to live in without needing to think about any crazy system spawning nonesense it could all be local. It ended up being a solid lesson in space conversion, going from world to local to camera space and understanding how those transformations stack.

Point-Cloud Soul

This was the effect that took the most R&D and ultimately taught me the most throughout the project.

The pipeline started with an Orbec Femto Bolt depth camera. In Touch Designer, we’d read in the depth data, clip away any excess space that wasn’t relevant, and then encode the positional data into the RGB channels of a texture. That texture got streamed directly into Unreal through Spout.

On the Unreal side, it was an exercise in converting 2D texture data back into a 3D particle effect by reading those RGB channels. Once I had an accurate point cloud recreation, the fun started.

I began by layering noise textures over the point cloud, which gave a predictable but visually interesting look. From there I went deeper, learning about the jump flood algorithm and SDFs. By converting the point cloud texture into a signed distance field, I had not only the positional data of the person’s silhouette from depth, but also a distance field I could use to push and pull particles. Combining the SDF with those earlier noise experiments produced this humanoid, spirit-like blob effect that the client ended up really liking. So we stuck with it.

Environmental Ambient Effects

Beyond the main systems, there were a couple of ambient effects worth noting.

The first was a gaze-reactive orb system, similar in principle to the void cursor. If you looked at a certain orb, it would expand and react more intensely. This doubled as a way to help users understand where the camera was currently pointed, since they didn’t have a traditional mouse or controller.

The second was more of a data-driven feature. The client and our art lead wanted a way to track how many users played through the full experience each day. As users completed their run and received a result, a colored gem tied to that result would spawn in a shared loading screen area. This was basically a simplified version of the cursor effect, but with the particles programmed to fly in circles and accumulate over time. The system would check whether a session had finished, and if so, added a gem to the stack. Once a group hit a certain count, maybe every five or ten, it would burst into a more colorful explosion effect.