For all images, responsible for shading only
Role: 3D Research and Prototyping
Tools: GLSL, C++, Substance Painter
Time: June - August 2024
Role: 3D Research and Prototyping
Tools: GLSL, C++, Substance Painter
Time: June - August 2024
Introduction
During my internship at Adobe, I worked on Project Neo, a 3D web app for 2D designers. I created a real-time stylized shader with hand-painted textures, exploring different NPR (non-photorealistic rendering) techniques such as 3D edge displacement, inverse hull outlines, and painterly blending. Compared to traditional mesh-based modeling softwares, Neo uses signed-distance fields (SDFs) instead, which opened up many opportunities to experiment with NPR techniques.
Try it here: projectneo.adobe.com
Video from Daichi Ito (@daichi_e) showing the final result in the web app
Conceptualization
The initial idea was inspired by the painterly style popularized in recent animated media such as Spiderverse and Arcane. We wanted to create a tool that would be easy for non-3D users to pick up on, with familiar controls and diverse visual results. Instead of emulating a specific visual look, I focused on emulating traditional painting/drawing techniques that would be intuitive and familiar to users.
To prevent screen-space flickering when the user rotated the screen, I focused on object space painterly effects.
There were three main design features:
1. Hand-painted textures - This helped provide a wide variety of looks quickly and worked with triplanar mapping.
2. Light and shadow - Inspired by painting techniques such as chiaroscuro, the shader relies on the distinct division between light and shadow, and the user can color each separately.
3. Breaking the "3D look" - Instead of keeping the model's surface flat, the shader distorts the edges of the model using textures.
While creating the shader, I also created the shader sliders that we would include on the front-end. As many of Neo's users are from a 2D background, I designed the slider controls to emulate digital brush creation programs (such as Photoshop's brush engine), with names such as brush stroke size, intensity, and color variation. The sliders and other controls were coded in React.
For finer control, the user can adjust the shader globally, as well as individually (per-object).
Overall, challenges included balancing technical/3D language with a user-friendly UI, optimizing texturing approaches to emulate a painterly look in real-time, and maintaining the shader's quality on a wide variety of models.
Object outline experiments with inverse hull.
