FMSG: Cyber: Learning Foundation Models for Manufacturing Design Automation

Northwestern University, Evanston IL

Investigators

Abstract

Large foundation models, such as GPT-4, LLaMA, CLIP, and BLIP-2, have demonstrated remarkable intelligence in interpreting user input and generating corresponding content. Such advancements reveal the potential for automating the manufacturing design process, as new foundation models could be created to understand a designer's intentions via inputs such as natural language, sketches, photos, or other modalities. These models can then automatically generate manufacturing designs in the form of CAD (Computer-Aided Design) models, and from which, further help the generation of manufacturing process instructions (e.g., G-code) by optimizing process selections and parameter settings. To realize this vision, this Future CyberManufacturing research project develops novel foundation models and learning methods that enable manufacturing design automation. It addresses unique characteristics and challenges in manufacturing designs, such as specific data types and stringent requirements (e.g., product specifications, manufacturing constraints, material selections), complex and diverse manufacturing processes, and difficulty in collecting large amounts of high-quality training data. The success of the project could bring transformative impacts by reducing design time, minimizing costs, and increasing product diversity and quality. Furthermore, a highly automated manufacturing design process could lower barriers for designers without significant manufacturing expertise, unleash their creativity, and increase labor participation in manufacturing. The end products would be more diverse and better suited to customer needs, thus benefiting society as a whole. The project develops a two-stage framework that includes novel foundation models and learning methods for manufacturing design automation. The first stage takes natural language and possibly additional images (drawings, sketches, photos, etc.) as input, and generates CAD models in textual representation as output, possibly followed by a manual model validation and revision step. This automatic generation of CAD models are enabled by novel methods for manufacturing-driven tuning of existing pre-trained language and vision models, multi-modal model fusion in manufacturing-specific representation space, design of a CAD generative decoder, and prompt engineering for model improvement. The second stage takes CAD models as input, along with optional textual hints (e.g., preferred manufacturing processes, cost constraints, etc.), and generates optimized manufacturing decisions, particularly the selection of processes and the setting of key parameters. These decisions, combined with existing tools, can help generate detailed manufacturing process instructions (e.g., G-code, additive manufacturing instructions). This stage includes novel methods for unsupervised learning of a process-agnostic foundation language model, supervised multi-task learning of process-dependent backends for optimizing process parameters, and reinforcement learning for process selection and further improvement of the input CAD model. This Future Manufacturing research is supported by the Computer and Information Science and Engineering Directorate's Division of Computer and Network Systems (CISE/CNS) and the Social, Behavioral and Economic Sciences Directorate’s Division of Social and Economic Sciences (SBE/SES). This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.