Understanding How 7 Billion Data Points Create a 3D Factory Model

Clients in the chemical industry frequently approach the Akkodis process engineering division, asking for engineering expertise to upgrade and modernize their aging production facilities. The Akkodis answer involves providing an accurate digital model of the facility “as built.”

However, these facilities have not yet arrived in the digital age. They often have outdated paper blueprints that fail to represent the actual state of the plant. In such cases, a LiDAR scan of the existing equipment can create a digital mock-up, the foundation for any redesign work.

Point Cloud Processing: Turning LiDAR Data into CAD Models

LiDAR is a proven technology for measuring distance with a rotating laser. It is used to make 3D maps of terrain and buildings and is a main sensor component in vehicle driver assist systems.

The challenge in creating a LiDAR-based 3D image of a chemical production facility is translating the vast number of LiDAR point clouds into CAD drawings. Advanced algorithms and CAD model automation techniques enable faster, more efficient upgrades. LiDAR point clouds are data points collected for a geographical area, terrain, building, or space. 

An analogy can be drawn between the syntax and semantics of words: a word may have different meanings depending on the context in which it is written or spoken. The same applies to 3D factory models generated from point cloud processing: a clever algorithm may recognize a cluster of points as a cylinder (the syntax).

However, the same cluster may be interpreted by a process engineer as a pipe, flange, or pressure vessel, depending on the surrounding context (the semantics). Currently, even with state-of-the-art algorithms, interpretation is done manually by a human.

The process engineering team wondered whether it could create a digital assistant that could automatically or semi-automatically identify the components of a production line, such as valves, flanges, and motors. A tool that could recognize key components would allow the company to offer clients a more competitive solution to modernization projects.

The team turned to Akkodis Research for help, asking whether it could build a tool pipeline capable of processing 3D point cloud scans from complex facilities. The goal was to automatically extract and analyze information relevant to the engineers redesigning and updating the facilities.

“We got to work,” said Manuel Reis Monteiro, R&D program manager at Akkodis in Germany. “Luckily, I could recruit a great internal team to contribute to the project. They’re doing an excellent job. Although this is still a work in progress, we’ve come far already.”

Using LiDAR Data for Identifying Components in Chemical Plants

The team's dataset consists of 160 LiDAR scans of a chemical plant in Germany manufacturing Montan wax. This base product is used to make polishes, paints, and lubricants. Each LiDAR scan contains 44 million points, for a total of around 7 billion data points.

Monteiro said that the Akkodis Research team will soon be ready to send the freshly developed tool pipeline to their processing engineering colleagues for testing and feedback.

The process engineers are looking forward to getting an AI-powered digital assistant to help them translate point clouds into valves, flanges, pipes, and motors. This approach enables faster CAD model automation for complex industrial systems. According to business manager Angela Marin-Betancourt, the concept is appealing from a business angle.

“In Europe alone, there are many older chemical plants built decades ago that have only blueprints of their production lines,” said Marin-Betancourt. “There is significant business potential in offering cheaper solutions for recreating ”as-built” models based on LiDAR scans that go beyond CAD model reconstruction.”

Monteiro admits that the project poses quite a challenge for the Akkodis Research team, which consists of experts from several domains. This is especially true since academic research is just starting to explore the integration of point clouds and GenAI. The issue is aligning the point cloud information with the Generative AI (LLM). Point Clouds are 3D data, and a GenAI must have a spatial understanding of them.

“Conventional algorithms can recognize basic geometric shapes like a cylinder, a plane, or a torus. It is common practice in the industry to take humans to reconstruct the CAD model from those geometries.” Monteiro said.

“We want more. We want to add AI and engineering knowledge to that. The idea is to refine the geometric primitives into engineering components so the software can identify items such as flanges with bolts and nuts, welding, straight and bent pipe sections, and more. The goal is to have a system that can automatically generate a CAD model from the LiDAR scans, with all the key components of a production line identified.”

Emplyoing AI for Efficient Point Cloud Classification

The solution must make sense not only from a technical but also from a business standpoint. There is no point in introducing a technology that makes things more expensive. Therefore, instead of training a complex Machine Learning model, the team is using “old-fashioned” artificial intelligence based on heuristic search, using heuristics as rules to classify points.

“We have used heuristics successfully in many Autonomous Driving systems and found them to perform very well in this context,” Montero said. “The big advantage is that you don’t need to train the system, which means you can develop and implement it quickly. We’ve got great results from that approach: We’ve saved time and effort and can concentrate on more advanced features like geometry recognition. And we’ve successfully got all the basic geometries out of the point cloud.”

We want to have a system that can automatically generate a CAD model from the LiDAR scans, with all the key components of a production line identified.

Manuel Reis Monteiro, Akkodis R&D Program Manager

ChatGPT and RAG for AI-Driven Plant Analysis

The team is exploring doing analyses based on general knowledge of process engineering and specific knowledge of the actual plant being scanned. Although the LLM already contains a lot of process engineering knowledge, it is not focused on the particular plant the team is working with. To achieve this, they have decided on a setup that combines ChatGPT with RAG (Retrieval Augmented Generation) systems.

Montero said, “A Generative AI needs a knowledge extension through a RAG system where we can explain what Montan Wax is, what main raw products and chemicals are required for its production, what chemical processes take place, and so on.”

“We can also provide a textual description of the “as-built” state of the plant to make it easier for the GenAI to infer a classification of objects.”

Marin-Betancourt, convinced of the potential of the new tool, added: “Imagine an assistant that can recognize the components of a production line and start putting labels on them: This is a pump, this is a flange, this is an electric motor. Of course, a skilled engineer is still required to check and perform quality control of the CAD model. But that will take much less time than doing everything manually.”