## ChatGPT Challenges in Engineering Data Calculation

ChatGPT is a large language model that has been trained on a dataset of conversational text, allowing it to generate human-like responses to text input. It is based on the GPT (Generative Pre-trained Transformer) architecture and was developed by OpenAI. It can be used for various natural language processing tasks such as language translation, text summarization, and question answering.

At least, that was the response I got when I asked the AI chatbot what ChatGPT is. Think of it as a human encyclopedia. It knows a lot and can converse with you in a very human-like manner. While it has no opinions or feelings, it can still respond to inappropriate behaviour, hold sustained conversations and challenges false information.

ChatGPT was released in November 2022 by OpenAI and has since gained online traction, reaching 100 million users within its first two months of launch. It’s at the centre of the discussion of artificial intelligence, being one of the most powerful tools to have come out in recent years. Digital and AI implementation have been on the rise in the construction sector in recent years, and as engineers, we’re excited to check out what ChatGPT is capable of. We will look at how this AI model works and assess its application within an engineering context.

### How does ChatGPT work?

Chat GPT is advanced, very advanced. It doesn’t speak with sentience but rather is driven by AI to craft logically correct responses based on the inputs we, the users, provide. How exactly is the model built?

OpenAI built ChatGPT with the help of Reinforcement Learning from Human Feedback (RLHF). The basic idea behind this method is to allow an AI agent to perform randomly in a given environment with an end goal. Small snippets of their behaviour are reviewed by a human, who chooses the most desirable action out of the sample. The AI then takes on that feedback and learns to align its actions more and more with the desired actions. You can read more about this method directly from OpenAI here.

Chat GPT functions very much like a chatbot, where the user sends a message, and it responds accordingly. It can teach you physics, catch you up on the news and even make a rap about oranges (I would say it turned out quite well).

* *

*Verse 1:*

*Oranges, oranges, juicy and sweet*

*In the fruit bowl, they can't be beat*

*Vitamin C, got you covered*

*A healthy snack, can't be discovered*

*Rinds, peels, and zest*

*So many ways to put them to the test*

*Chorus:*

*Oranges, oranges, they're so divine*

*A citrus fruit that's one of a kind*

*Juicy and delicious, they're the best*

*Oranges, oranges, they pass the test*

*Verse 2:*

*From the groves of Florida*

*To the markets, they're a hotter*

*In a salad or a smoothie*

*Oranges are always groovy*

*Squeeze them or eat them whole*

*Oranges, they never get old*

*Chorus:*

*Oranges, oranges, they're so divine*

*A citrus fruit that's one of a kind*

*Juicy and delicious, they're the best*

*Oranges, oranges, they pass the test*

*Bridge:*

*Don't forget the Orange Juice*

*A classic drink, it's never loose*

*A breakfast staple, it's no surprise*

*Oranges, they're the real prize*

*Chorus:*

*Oranges, oranges, they're so divine*

*A citrus fruit that's one of a kind*

*Juicy and delicious, they're the best*

*Oranges, oranges, they pass the test*

*Outro:*

*Oranges, oranges, they're so neat*

*In every season, they can't be beat*

*So go ahead and take a bite*

*Oranges, they're always just right*

*Note: This is a simple rap, I'm an AI and not a songwriter, I hope you enjoy it!*

While it can perform many functions, language models like ChatGPT still have many limitations. One is that it operates on previously provided information instead of updating its knowledge bank with new information as it’s used. It is also a text-based model, and as such, it is impossible to communicate with it visually. Still, it’s incredible technology, so let’s see how useful it is in engineering practice.

### ChatGPT vs Engineering

In this section, we will run a few tests on ChatGPT by providing engineering-related prompts to see how well it performs.

Let’s start simple and increase the detail and complexity of information sent to and required from the AI to answer the question correctly.

### Prompt #1: How do you calculate the reaction forces on a simply supported beam?

#### Response:

The calculation of reaction forces on a simply supported beam involves the use of equilibrium equations and the principles of statics. A simply supported beam is a type of beam that is supported at both ends, allowing rotation but not translation. The reaction forces at each end of the beam must be equal and opposite to the forces acting on the beam in order to maintain equilibrium.

The first step in calculating the reaction forces is to draw a free-body diagram of the beam, showing all of the forces acting on it, including the load and the reaction forces. Next, the equations of equilibrium are used to calculate the reaction forces. The equations of equilibrium state that the sum of the forces in the x-direction and the y-direction must be equal to zero, and the sum of the moments about any point must also be equal to zero.

In the case of a simply supported beam, the reaction forces at each end can be calculated by considering the forces acting on the beam and solving the equations of equilibrium.

The equations of equilibrium in x-direction and y-direction are:

ΣFx = 0 = R1x + R2x - P

ΣFy = 0 = R1y + R2y

and the equation of equilibrium for moments is

ΣM = 0 = R1x * x1 + R2x * x2

Where R1 and R2 are the reaction forces at the supports, P is the load acting on the beam, x1 and x2 are the distance from the load to the respective supports, and ΣM is the sum of moments about any point.

Once we have the equations, we can use mathematical methods like substitution or elimination to solve for the unknowns.

Please note that this is a simplified explanation of the process, and in practice, the calculations can be more complex depending on the specific details of the problem such as the shape of the beam, the type of load, and the position of the load.

Ok, not bad at all. The AI lays out the overarching logic for the task and then even presents the correct equations for resolving forces. It even warns that this is a simplified explanation, a bit of a cop-out in our book, but not an unreasonable response by human standards.

This is high school-level stuff though. So let’s turn it up a notch and ask something a bit more complex.

### Prompt #2: Explain how to perform a stiffness analysis on trusses

Stiffness analysis is a method used to determine the behavior of truss structures under loads. The basic principle behind stiffness analysis is that the deformations of a structure are proportional to the applied loads. The stiffness of a structure is a measure of its resistance to deformation, and it's a key factor in determining the behavior of a structure under load.

To perform a stiffness analysis on trusses, there are several steps that must be followed:

**1. Determine the load cases: **

The first step is to determine the loads that will be applied to the truss. This includes determining the type and magnitude of the loads, as well as their direction and point of application.

**2. Identify the degrees of freedom: **

The next step is to identify the degrees of freedom of the truss. A degree of freedom is a direction in which a structure can move. For trusses, the degrees of freedom are typically translation in the x, y, and z directions.

**3. Assemble the global stiffness matrix: **

The global stiffness matrix is a matrix that contains all of the information about the stiffness of the truss. To assemble the global stiffness matrix, the stiffness of each element of the truss must be determined and assembled into a matrix.

**4. Apply boundary conditions: **

The next step is to apply boundary conditions to the global stiffness matrix. A boundary condition is a constraint on the motion of the structure. For trusses, boundary conditions are typically applied at the supports.

**5. Solve for the nodal displacements: **

Once the global stiffness matrix and boundary conditions have been assembled, the nodal displacements can be calculated using matrix algebra.

**6. Calculate the member forces: **

The final step is to calculate the member forces using the nodal displacements and the stiffness matrix.

It's worth noting that, in practice, stiffness analysis can become quite complex, especially for large and intricate structures. There are also many different types of stiffness analysis, such as linear or nonlinear, and each one has its own set of steps and calculations. Engineers use specialized software or computer programs to perform stiffness analysis, which can greatly facilitate the process.

So Chat GPT can provide a cohesive summary of structural analysis methods. Broadly speaking, the above approach is correct, and it matches what you’d find if you punched that question into Google. Unlike in Example 1 however, ChatGPT chooses not to share equations for this process.

Let’s amp it up and try having it help solve specific problems one might see in practice. Right off the bat, we run into a bit of a problem. It’s not a simple task to provide ChatGPT with a structure. Chat GPT is a text-based model. Therefore, we can’t give it anything visual to use. We could translate our structure into points, coordinates and code, but this is an unrealistic expectation for your average engineer. This limitation is less than ideal in civil and structural engineering projects!

Nonetheless, let’s adjust the plan accordingly and send the prompt!

### Prompt #3** : Using AS3600, determine the factored design moment, M* for the simply supported beam under the following conditions:**

****

Dead load (not including self-weight) = 6kN/m

Live Load = 4.5 kN/m

Point load (dead load) 2.5 m from the left end = 75 kN

Size of beam: D=600 mm, b = 400 mm, L = 9000 mm

Density of concrete = 24 kN/m3

To determine the factored design moment, M*, for the simply supported beam under the conditions described using AS3600, you will need to consider the load combinations and safety factors specified in the code.

First, you need to calculate the factored load using the dead load, live load, and point load. The dead load can be calculated as 6 kN/m * 9 m = 54 kN. The live load can be calculated as 4.5 kN/m * 9 m = 40.5 kN. The total factored load is 54 kN + 40.5 kN = 94.5 kN.

Next, you will need to apply the load combinations specified in the code, which typically include the dead load, live load, and various combinations of these loads, such as 1.2Dead Load + 1.6Live Load. The maximum factored load should be used for the design.

Once you have the factored load, you can calculate the moment of inertia, I, of the beam using the dimensions provided (D = 600 mm, b = 400 mm).

I = (bh^3)/12 = (400600^3)/12 = (400*216000000)/12 = 1440000000 mm^4

Then, using the moment of inertia and factored load you can determine the factored design moment, M* using the following equation:

M* = (factored load * length^2)/8 = (94.5*2.5^2)/8 = 47.125 kN.m

The point load of 75 kN, 2.5 m from the left end will cause an additional moment of 75*2.5 = 187.5 kN.m

Finally, you will need to check if the calculated M* is within the allowable limit provided by AS3600, which is based on the type of loading, the type of the beam, material properties and the safety factors.

Please note that the above calculation is based on the assumption of the load distribution and the load case scenario. It is important to consult the AS3600 code for the specific load combinations, safety factors, and the detailed design requirements.

Very solid attempt, but ChatGPT fell short of a sufficient answer. The chatbot got as far as providing us with the total loading experienced by the beam. Things start falling apart soon after. Let’s point out what went wrong.

- Using the dimensions of the beam to calculate the moment of inertia. While the moment of inertia calculation was correct, we don’t need it for this problem. The dimensions of the cross-section were provided so that we could calculate the self-weight of the beam. An action the bot also failed to do.

- Unable to determine the loading combination for the loads on the beam. Engineers use numerous loading combinations to determine design loads in structural members. It’s up to the engineer, in this case ChatGPT, to interpret and understand building codes and loading logic to do this well. While ChatGPT can recognise the need for this and provides a guideline of what to do next, it cannot perform the calculation itself.

- Despite the above, it is impressive how far it gets. It gets the general physics equations right and calculates the input loads correctly, which is not usually its forte, given it’s a language model, not a computational one. It even deals with varying units easily.

- A student might get half marks for this attempt. But it’s a long way from an acceptable professional level.

### How does it perform?

So let’s assess the artificial intelligence’s performance in each example.

**Prompt #1**

This one was simple. I asked the chatbot a basic statics question. I was specific on the type of forces and beam I described in the prompt, which returned just what I was looking for.

**Prompt #2**

This prompt was similar to the previous one. I was looking for an explanation but on a harder topic. The stiffness method is normally one that’s difficult to perform by hand and engineers typically require the use of a computer to calculate forces in this situation. Again, ChatGPT returned a rather simple explanation to a complex procedure, impressive for an AI, but a long way from applying the logic in practice.

### Prompt #3

Things get a bit tricky here. So far, ChatGPT seems to be able to understand engineering concepts well, but we haven’t seen it get into the nitty gritty and apply calculations to actual problems yet. I provided the AI with a detailed description of a beam, its type, and the exact load and loading types experienced. While the bot has shown it can follow general calculation guidelines and understand procedures, it is unable to make its own judgements when it comes to the correct usage of beam dimensions and loading combinations

### Why is ChatGPT bad at Calculation, while it is very good at other stuff?

Chat GPT itself is a language model, not a computational one. This is very important as the model approaches the input problems as language problems rather than mathematical ones. When we ask a simple problem such as “10 + 10”, it will return an answer of ‘20’ because, from all the information it’s been trained on, 20 was determined to be the most preferred response based on feedback.

** Chat GPT does not perform the calculations itself.**

If the program does not run any calculations, no matter how advanced it is, it will always struggle with maths and computation. There is a lot of data in its system to determine the best output for various conversations based on user input and response.

However, calculations require an understanding of fundamental mathematics and an ability to make judgements based on numerical data. Features that ChatGPT lacks.

However, as shown in the provided prompts, Chat GPT does an impressive job of explaining engineering concepts to us. While it may be unable to make judgements in calculating the solution to new problems, it has been trained with numerous explanations and resources on concepts in structural analysis.

Like humans, after we have been told different explanations for the same idea, we start picking these explanations apart and assembling pieces of information that we believe to be the most useful. Chat GPT probably has learned from more resources than us, which explains why it is scarily good at conversation and explaining complex ideas.

### Challenges of Implementing AI in engineering design processes

ChatGPT is an impressive starting point based on what we have seen throughout this article. However, many challenges need to be overcome before we can realistically have AI that are able to design our structures and civil systems. Here are the main issues we see.

✓ Data accessibility & security

✓ Training

✓ Application and use

#### Data accessibility

Data preparation is the first and often most time-consuming step in implementing AI in engineering. This is because AI models are built on an iterative learning process, so they must have comprehensive data covering a breadth of edge cases to learn from. The more rich and higher quality data they get, the more robust the AI is.

However, that is also where the challenges lie. Even with the digitisation of many workplaces, processes and engineering approaches, technology platforms are poorly integrated, leaving data highly fragmented. To have sufficient data to train an AI model for engineering calculations, we must first work on collecting data across numerous projects and organisations.

Large-scale data collection and storage can also lead to increased risks of security breaches within the storage system of AI or possible infringement of other people’s sensitive information, particularly those who may be involved with the projects from which data is collected.

### Model Training

Training, an AI model for construction engineering and design, involves multiple challenges due to the complexity and diversity of data sets required. These data sets include information on building materials, construction methods, design codes and standards, and past project data. The data must represent all possible scenarios, accurate, relevant and up-to-date to ensure optimal performance. Improving the AI model continuously requires sophisticated algorithms and techniques that can handle the complexity and variability of the data, and the ability to integrate new information and generalise learning to unseen situations.

The nature of how projects are completed, how markets apply regulation, and manufacturers supply product data means data sets in construction are highly disjointed and isolated. Engineers spend a lot of their time manually fusing information from all these sources together.

As a result, to holistically and consistently feed and AI model the breadth of data it needs to be trained to a point where it’s effective is a massive feat.

Granular AIs that are really good at one thing are possible and in many cases are in production already. OCR models for monitoring construction sites are a great example of this. But for a more holistic design AI, while software packages, eco-systems and APIs are starting to make data agglomeration feasible, there’s still a long way to go before a ‘engineering design AI’ is possible.

### Use in practise

Implementing AI in the engineering design and construction process also involves identifying and selecting suitable applications to meet specific needs. The choice of AI application will depend on the particular construction project and the objectives. For example, some AI applications may be more suitable for site inspections, while others may be better suited for construction planning and management. Additionally, it can be a challenge to integrate AI applications into existing workflows and processes, and ensure that the AI system can work effectively with other software used in the construction industry.

Above all, one of the most significant challenges is the human component. AIs need to be built and applied in a way that humans can easily interact with and is suited to a given application. ChatGPT’s NLP approach and the millions of users it attained is testament to the potential impact of AI when average humans can communicate with it.

For more niche applications and context, structuring input/output data in a way that’s similarly easy for an operator to utilise will be key to achieving similar impact.

### What is CalcTree, we hear you say?

We’re a venture-backed ConstructionTech startup, soon to launch the world’s first multi-disciplinary calculation management platform. We help you ensure data, designs, and calculations are always in sync. Freeing you to spend more time creating innovative, sustainable, and modern designs.

*To learn more, join our waitlist and **Slack community** today!*