aalto1 untyped-item.component.html
Multi-agent generative modeling of electro-mechanical system components
Loading...
URL
Journal Title
Journal ISSN
Volume Title
School of Engineering |
Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Authors
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
72
Series
Abstract
Generative and computational synthesis methods offer an unbiased and efficient way for design space exploration. However, evaluation of the generated concepts remains a challenge. Simulations are frequently the most practical method for this purpose, but require significant domain-expertise, temporal and computational resources. Large Language Models (LLMs) offer an opportunity for allowing rapid and efficient evaluation through their expertise in code generation and problem solving. However, their inability to perform deeper scientific and logical reasoning prevents them from being reliable and independent tools in this context.
This thesis proposes an LLM-based multi-agent system and explores its feasibility to generate component-level lumped-parameter models for this purpose. The system divides the overall modeling workflow into LLM-based agents with well-defined roles and responsibilities. These include; the architect for formalizing the problem statement, the engineer for writing simulation code, and the scientist for evaluating the code’s scientific logic. A Python-based modeling template is developed to ensure modularity and standardization through bond-graph based port structures. The syntactic correctness of the code and compliance to the template is ensured by validating it through pre-defined unit tests.
Testing on electro-mechanical component modeling tasks reveal a low first pass score of 11% and an overall accuracy of 26%. The failures were generally propagated by the architect struggling to infer task-relevant information from the user. The engineer consistently produced compliant and correct code but struggled with the scientific basis, particularly for tasks involving non-standard modeling logic. The scientist displayed subpar performance and failed to detect several scientifically incorrect parts in the generated models.
The results reinforce the strengths of LLMs in syntax-driven coding and their limitations in scientific reasoning. They also underpin the importance of requirement formalization to understand user intent. Aside from the system, this thesis contributes to the broader understanding of LLM performance in different roles within a structured engineering workflow. Future work could focus on dialogue-based requirement elicitation, advanced prompting techniques to improve scientific reasoning, and expansion of the modeling template to acausal and non-sequential system design.