Opportunities for large language models and discourse in engineering design

In this paper, we argue that foundation models such as LLMs can be used for creative reasoning tasks in the engineering design process, complementing and integrating existing computational methods such as topology optimization.

First, we provide engineers with a summary of the recent advances in NLP and outline which aspects of engineering design have been digitized thus far (Section 2). In Section 3, we place goal-oriented, argumentative discourse at the center of the product development process (see Fig. 1) and propose making the reasoning steps explicit in the form of a new digital artifact. On this basis, we describe how LLMs and multimodal foundation models can assist in the design discourse (Section 4) and outline interesting directions of future research (Section 5). The presented ideas are transferable to other contexts in which creativity and reasoning play an important role, such as scientific discovery in general.

Lexical databases [36–38] and stopword lists [39] for technological vocabulary and jargon have been proposed, as have engineering-related ontologies [40–42]. With ontologies come knowledge graphs. However, there has been a lack of specialized engineering knowledge graphs thus far [43]. Only recently, Siddharth et al. [44] build a knowledge graph using patent claims. With a trend towards industry 4.0 and digital twins, the subject of the semantic representation of technological knowledge will probably be increasingly addressed in the future. NLP methods, which have been less applied in the engineering sciences compared to the biomedical and material ones, have become increasingly popular recently. In design research, NLP has been applied to requirements extraction, ontology construction, patent analysis, and more [45].

we concentrate on the design process itself as a complex, iterative, and dynamic reasoning process and situate recent advances in NLP and machine learning in a superordinate framework.

has not included the creative and argumentative process of the product development process itself. In the following, we argue that this process could be digitized and partially automatized next,

Until now, however, computer-aided engineering, as practiced in industry, has not included the creative and argumentative process of the product development process itself. In the following, we argue that this process could be digitized and partially automatized next, and outline how this can be achieved.

Many steps in the product development process are performed using computation and are not based on human thought alone. However, humans are needed to integrate these computational processes, be they calculations, simulations, or optimizations, into a meaningful superordinate product development process. Human thought and world knowledge is required to reduce the solution space in advance and come up with original ideas

Solving engineering problems requires an argumentative discourse. As such, argumentation is inherent to the product development process. Experiments and calculations, etc., inform the discourse to provide necessary information. Nevertheless, argumentation is rarely given a lot of attention, perhaps because it is hidden,

Having described that a goal-driven, argumentative discourse is at the core of the design process, we argue that it should be represented as a digital artifact.

Representing the argumentative discourse as a digital artifact would improve the documentation of the design process. Instead of only archiving the results of process steps (e.g., CAD files or the results of simulation runs), the reasoning process is documented and hence archivable. For a past development process to be efficiently used for the development of a new product generation, past decisions and alternatives must be accessible. Having the reasoning process explicitly documented makes past design decisions traceable. Furthermore, making the reasoning process explicit could improve collective reasoning and therefore collaborative design. Finally, it would allow for machines to participate in the reasoning process, which the next section covers.

the potential for machines to participate in the reasoning process. LLMs and related multimodal models have several characteristics that suggest that they can be successfully applied for this purpose.

To increase the interpretability and accuracy of LLMs, approaches such as scratchpad [66] or chain-of-thought prompting [24] steer the models towards the generation of intermediate steps. Complementary to the aforementioned approaches of generating intermediate steps, models can be guided to imitate certain patterns of thinking or to follow a logical flow by embedding calls to LLMs into a framework with a predefined causal structure. In such frameworks ‘‘querying a language model becomes a computational primitive’’ [67]. For example, answering questions in steps adhering to formal logic yields interpretable reasoning traces and reduces the ‘‘hallucination’’ of facts [67]. Similarly, formalizing the engineering design discourse within a framework of stages and recurring components will help safeguard the models and increases the detail at which reasoning processes can be documented and verified.