Researchers Unveil LegoGPT AI Model That Can Build Physically Stable Design of Lego Structures

LegoGPT, a brand new synthetic intelligence (AI) mannequin that may generate three-dimensional (3D) Lego construction designs, was unveiled by researchers not too long ago. The new AI mannequin is an open-source challenge to find out whether or not AI fashions can generate constructions which can be in step with real-world physics and are persistently secure. The researchers have shared particulars of how the mannequin was constructed, in addition to have made the dataset out there to the general public area. The AI-generated Lego constructions had been additionally examined by people and robots to substantiate the steadiness of the constructions.

LegoGPT AI Model Was Built on LLaMA-3.2-Instruct

In a put up, researchers from Carnegie Mellon University detailed the LegoGPT AI mannequin. The giant language mannequin (LLM) can generate a Lego construction from a textual content immediate, making certain it’s bodily secure and buildable. The open-source mannequin is obtainable to obtain and use on GitHub with a permissive MIT licence.

Users can immediate the mannequin to design a “streamline elongated vessel” or a “backless bench with armrest,” and it might generate a design that not solely matches the outline but additionally might be positioned upright with out the construction collapsing.

This is feasible as a consequence of two parts that make LegoGPT — the bottom AI mannequin and a stability evaluation system. For the bottom mannequin, the researchers used a fine-tuned model of the Llama-3.2-Instruct with one billion parameters. This was paired with Gurobi, a mathematical optimisation solver, that runs stability evaluation for every generated construction.

Alongside constructing the refined structure, the researchers additionally created a dataset to coach the mannequin on Lego constructions. Dubbed StableText2Lego, it’s a dataset containing greater than 47,000 Lego constructions of over 28,000 distinctive 3D objects. Each construction is accompanied by detailed captions, design code, and fashions.

To confirm that the generated constructions are certainly secure, the researchers additionally examined them with a twin robotic meeting. The meeting was tasked with recreating the designs and testing whether or not they might stand upright. Some of the designs had been additionally recreated by people to see the influence on stability if much less dextrous palms had been concerned. The analysis paper claims that 99.8 % of all constructions handed the steadiness take a look at.