Artificial intelligence (AI) and machine learning are revolutionizing software applications across industries. In image editing, Adobe Photoshop now leverages Firefly generative AI, while in software development, GitHub Copilot enhances code quality and productivity. The game development sector, however, has yet to fully harness AI's potential, particularly in node-based systems for shader creation.
Node-based systems enable developers to create shaders, visual effects, and complex systems including animations, procedural textures, and game mechanics. Rather than writing code manually, developers can utilize predefined code blocks to achieve desired results. However, establishing the foundational structure of node-based systems remains a complex and time-intensive process
This thesis investigates the application of advanced language models – specifically Google's Gemini 1.5 and OpenAI's GPT-4o – in developing a generative AI system for node-based shaders. Through the implementation of prompt engineering, retrieval-augmented generation, and fine-tuning techniques, we propose an AI-assisted shader graph system. This system will autonomously generate base nodes and establish graph connections in response to artists' natural language prompts.
Our approach aims to accelerate the iteration process and reduce time spent on basic setting manipulation. For example, given an artist's input such as 'Create an explosive particle effect with 2,000 particles transitioning from red to black over 2 seconds,' the system initiates the node generation process. Artists can subsequently refine their input or manually adjust the generated nodes to achieve optimal results.
This research demonstrates the potential of language models to power a generative AI system for node-based shader development, paving the way for significant advancements in efficiency and creativity within the game development industry. By bridging the gap between artistic vision and technical implementation, we anticipate a profound impact on workflow optimization and the accessibility of complex shader creation.
