#062 - Machine Learning for Engineers | When and How Should We Use it?
A Practical Guide to Integrating ML into Professional Engineering Workflows
Machine learning (ML) is becoming more approachable and useful for engineers.
This article introduces practical applications of machine learning specifically tailored for engineers. You'll learn how ML aligns with engineering intuition, explore realistic structural examples using PyTorch, and gain a structured approach for assessing ML's suitability in your projects.
I’ve discussed this before in relation to Linear Regression models, so check that out for a deeper dive into the nuts and bolts if you are interested in getting started.
The space has evolved since then, as ML gains more traction.
One of the most fascinating aspects of ML is the realization of how deeply it aligns with engineering intuition. Consider the intuitive understanding an engineer develops regarding the factors influencing design. Aspects like feature engineering, scaling, and learning rates, which might seem esoteric, are in fact reflections of the inherent understanding we build through practical experience.
Once you begin to explore the theoretical underpinnings, you'll find that this intuition is often validated by rigorous mathematical and statistical methods. It’s a rather satisfying confirmation, at least in my view.
This also sparks other deeper philosophical questions. If you can capture intuition in an algorithm, how long until we can capture thoughts/emotions etc.
The singularity's arrival is, apparently, imminent – accompanied by either the collapse of civilization or our ascension to a higher plane of AI-assisted existence.
Thankfully, these grand pronouncements are beyond the scope of this article, and my expertise, unless I’ve had a few pints. 🍻
Understanding Machine Learning
ML involves algorithms that learn from data, continuously improving performance without explicit programming. Although sometimes viewed as "black boxes," these methods systematically uncover useful patterns and insights.
My own journey into machine learning, and I want to be clear that I'm very much still on that journey – I'm certainly not claiming to be an expert – began with exploring libraries like Scikit-learn. I focused initially on fundamental concepts like linear regression and classification, which are surprisingly approachable. This provided a solid foundation, and I encourage anyone curious to start there. From that base, I expanded into more complex frameworks.
If you're looking for resources:
Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow by Aurélien Géron (O'Reilly)
Deep Learning with Python by François Chollet (Manning Publications)
They strike a good balance between theory and practical application. For a structured online learning experience:
Machine Learning Specialization course from deeplearning.ai is excellent.
The point I want to emphasize is that this isn't inaccessible material. It's surprisingly approachable, and the learning curve, while real, is manageable.
My own experience, moving beyond the basics and applying PyTorch to real-world engineering in the hydropower sector (structural design optimization, hydraulic modeling, financial and energy forecasting), has convinced me of ML's versatility and value. I'm sharing this not as an expert, but as a fellow engineer who's seen the potential firsthand and believes strongly that more of us can and should explore these tools.
Types of Machine Learning
Supervised Learning: Models trained on labeled datasets to predict known outcomes, such as concrete strength or structural damage classification. Common applications include structural monitoring and maintenance planning. Vision models are now using this for damage detection. You can try building such tools yourself with Pytorch and YoloV8.
Unsupervised Learning: Identifies hidden patterns or clusters within unlabeled data, useful for material categorization and detecting construction anomalies.
Semi-Supervised Learning: Utilizes both labeled and unlabeled data, efficiently addressing tasks like detecting structural issues from limited annotated images.
Advanced Learning Methods:
Active Learning: Selectively trains on the most informative data to maximize efficiency.
Transfer Learning: Adapts pretrained models from one task to related tasks, such as using generic image recognition models to detect specific engineering defects.
Reinforcement Learning: Models learn through reward-based systems.
Self-Supervised Learning: Uses portions of data to predict other segments, enhancing pattern recognition capabilities.
Comparing ML with Traditional Methods
Traditional engineering methods, including finite element analysis and empirical formulas, depend on explicitly predefined assumptions. These are robust but less flexible when handling complex, real-world variability.
Despite initial skepticism, the growing efficiency and accuracy of ML present clear advantages. Its rapid adoption across fields like finance, software, and other engineering disciplines illustrates its growing acceptance, especially among younger professionals. I can see the usage rates for younger engineers in my office for our internal LLM tools and they absolutely dominate the usage. A small sample size yes but I’d wager that it’s representative! This is only going to accelerate whether people like it or not.
However, embracing ML requires considering its energy and resource demands, both from a personal perspective and from a broader business and industry perspective. As Vaclav Smil notes:
“We must always consider the energy and resource costs of these new technologies, ensuring we are not simply trading one set of problems for another.”
Example: Predicting Optimal Beam Depth Using PyTorch
The following PyTorch example (code: GitHub Gist) is not intended as a replacement for established structural engineering principles. Instead, it provides a focused illustration of how machine learning can supplement traditional calculations, particularly when dealing with complexities that idealized formulas struggle to capture.
Our approach simulates complex structural behavior by introducing realistic variations in material properties, support conditions, load distributions, and geometric imperfections—factors that engineers typically evaluate using FEA software.
The visualizations reveal how different ML models capture this complexity.
Notice in the span and load plots how ML models closely follow the traditional formula's curve but with subtle differences that reflect the additional complexity they've learned. The error distribution further demonstrates how deeper models minimize prediction errors.
The visualizations demonstrate the ML models' ability to learn the complex, non-linear relationship between span, load, and beam depth. Unlike traditional formulas that rely on nominal values, our ML approach, trained on synthetic data generated with realistic variations, captures the nuances arising from:
Load Variations: Uncertainties in the actual applied load (±5% in this simulation, based on some uncertainties regarding ancillary loads and equipment).
Material Strength Variations: Variations in the steel's yield strength (±3% in this simulation, reflecting steel grade tolerances).
Geometric Variations: Manufacturing tolerances in the beam's width (±5% in this simulation).
The close alignment of the deeper non-linear models with the simulated FEA results (particularly evident in the "Predictions vs. Actual" and "Error Distribution" plots) is not a coincidence. It highlights ML's strength: identifying patterns in complex, multi-variate data without being explicitly programmed with the underlying structural mechanics equations. The model learns these relationships from the data. Of course you can include your capacity calculations, checks or processes as you wish, depending on your objective.
This capability translates to several practical benefits for engineers:
Rapid Design Space Exploration: Quickly evaluate a wide range of design options, considering the variations mentioned above. I often like to explore a wide range of options or structural behaviours to get an intuitive sense of a structures response.
Enhanced Accuracy: Capture subtle behaviours that simplified formulas might miss, leading to more realistic predictions or at least calibrated for the uncertainties of your project, maybe you're in the early stages with many unknowns.
Robust Design: Develop designs that are less sensitive to real-world variations and uncertainties. You can measure, test and validate such cases.
Data-Driven Iteration: Use previous FEA or experimental results to inform and improve future designs.
It's important to remember that this example is a simplified demonstration.
However, the principle of using ML to create surrogate models – computationally inexpensive approximations of more complex simulations – is a well-established and rapidly growing practice in engineering design optimization.
Commercial software packages from Altair and Ansys, among others, provide tools to facilitate this, and it remains an active area of research.
Creating a concise, practical example of ML for structural engineering, without oversimplifying or getting lost in technical details, was more difficult than I anticipated. This represents my best effort. Did it effectively convey the key ideas? Your honest feedback is crucial; please share your thoughts in the comments below.
A Practical Framework for Evaluating Machine Learning in Engineering
Machine learning is just another tool, and like any tool, its effectiveness depends on appropriate application.
This framework provides a structured, step-by-step process for engineers to determine if ML is a suitable solution for a given engineering problem, and if so, how to approach it strategically.
Phase 1: Problem Suitability Assessment
This phase focuses on whether ML is even a potential solution.
Precise Problem Definition (The "What"):
Articulate the Specific Engineering Objective: Go beyond generalities. Instead of "improve bridge design," specify "predict the max deflection of concrete bridge deck, girder 3 at mid span under specific loading conditions with an accuracy of ±X mm."
Classify the Problem Type: Is it:
Prediction (Regression): Predicting a continuous numerical value (e.g., stress, strain, deflection, temperature, cost).
Classification: Assigning an instance to a predefined category (e.g., damage type, failure mode, material type, suitable/unsuitable design).
Optimization: Finding the best set of parameters to achieve a specific objective (e.g., minimize weight, maximize strength, minimize cost). You can rank parameters depending on importance.
Anomaly Detection: Identifying unusual patterns or outliers (e.g., detecting structural defects, identifying sensor malfunctions or signal noise).
Pattern Discovery (Clustering/Dimensionality Reduction): Finding hidden relationships or groupings within data (e.g., identifying common failure modes, grouping similar structural components, deduce abnormal load paths or unintended effects).
Validation: Can the ML solution support or validate existing analyses.
Quantifiable Success Metric: Define how you will measure success. What level of accuracy, precision, or improvement is required for the ML solution to be considered valuable? This is important for evaluating the model's performance later.
Go/No-Go Decision Point: If the problem cannot be clearly defined in one of these terms, or if a quantifiable success metric cannot be established, ML is probably not a viable solution.
Feasibility Check (The "How"):
Traditional Methods Baseline: Can the problem be solved adequately using traditional engineering methods (e.g., hand calculations, established formulas, FEA)? If so, what are the limitations of these methods that ML might address? (e.g., computational time, inability to handle complex geometries, difficulty incorporating real-world variability).
Data Requirements (Preliminary): Based on the problem type, estimate the kind and approximate quantity of data that would be needed. (See Phase 2 for a deeper dive into data). This is a preliminary assessment.
Resource Constraints: Realistically assess the available resources:
Time: How much time can be dedicated to developing and implementing an ML solution?
Expertise: Does the team possess the necessary ML expertise (data science, programming), or will external consultants be required?
Computational Resources: Are sufficient computing power and software available for data processing, model training, and deployment? You can do a lot with Python and Pytorch. I have yet to run into a hardware bottleneck. You can always scale your efforts to your capabilities.
Budget: What is the budget for data acquisition, software, hardware, and personnel?
Go/No-Go Decision Point: If traditional methods are entirely sufficient, and the potential benefits of ML do not outweigh the resource constraints, stop here. ML is not always the answer.
Phase 2: Data Evaluation
If Phase 1 suggests ML is potentially viable, this phase focuses on a rigorous assessment of the data.
Data Audit (The "Reality Check"):
Source Identification: Where will the data come from? (e.g., FEA simulations, experimental measurements, sensor data, historical records, publicly available datasets).
Data Availability: Does the data already exist, or will it need to be generated or collected?
Data Quantity: Estimate the actual amount of data available or obtainable. Is it sufficient for the chosen problem type and model complexity? (General rule: more complex problems and models require more data).
Data Quality Assessment: This step is so important, I can't overstate it. Examine the data for:
Accuracy: Are the data values correct and reliable?
Completeness: Are there missing values? If so, how will they be handled (imputation, deletion, interpolation)?
Consistency: Are the data values consistent across different sources and time periods?
Relevance: Does the data directly relate to the problem being addressed? Are the features (variables) likely to be predictive of the target variable?
Bias: Does the data represent the full range of conditions and scenarios relevant to the problem? Biased data will lead to biased models.
Noise: How much random error or irrelevant information is present in the data?
Data Dictionary/Metadata: Document the meaning of each variable, its units, and any relevant information about how the data was collected.
Data Acquisition/Cleaning Cost: Realistically estimate the time and effort required to acquire, clean, and preprocess the data. This is often the most time-consuming part of an ML project.
Integrity check: After data cleaning, assess that the core characteristics and usefulness is still present.
Go/No-Go Decision Point: If the data is unavailable, insufficient in quantity, of poor quality, irrelevant, or prohibitively expensive to acquire/clean, reconsider the ML approach. Garbage in, garbage out is a fundamental truth.
Phase 3: Model and Implementation Strategy
If the problem is suitable and the data is adequate, this phase focuses on the technical implementation.
Model Selection (The "Algorithm"):
Problem Type Match: Choose an ML algorithm appropriate for the problem type (regression, classification, etc.). Consider:
Linear Regression: For simple linear relationships.
Polynomial Regression: For non-linear relationships.
Support Vector Machines (SVM): For classification and regression.
Decision Trees/Random Forests: For both classification and regression, often interpretable.
Neural Networks: For complex, non-linear relationships (require more data).
Clustering Algorithms (k-means, hierarchical): For grouping similar data points.
Dimensionality Reduction Techniques (PCA, t-SNE): For reducing the number of variables.
Interpretability vs. Accuracy Trade-off: Determine the importance of model interpretability. If understanding why the model makes a particular prediction is crucial (e.g., for regulatory compliance or gaining engineering insights), simpler, more transparent models (e.g., linear regression, decision trees) are often preferred, even if they are slightly less accurate than "black box" models (e.g., deep neural networks).
Computational Cost: Consider the computational resources required to train and deploy different models. More complex models typically require more computational power.
Pre-Trained Models: While a wide range of pre-trained machine learning models exist, and transfer learning can be a valuable technique, most engineering design optimization problems are too specific and constrained to be solved directly by off-the-shelf solutions. Engineers typically need to carefully define their objectives, generate or collect relevant data, and customize or fine-tune models significantly to achieve useful results. Pre-trained models can provide a helpful starting point.
Model Training and Validation (The "Iteration"):
Data Splitting: Divide the data into training, validation, and test sets. The training set is used to train the model, the validation set is used to tune1 hyperparameters (model settings), and the test set is used for a final, unbiased evaluation of the model's performance.
Cross-Validation: Use techniques like k-fold cross-validation to ensure the model generalizes well to unseen data.
Hyperparameter Tuning: Optimize the model's hyperparameters using the validation set. This is a more advanced rabbit hole. Most basic models are likely more than enough for most engineering optimization efforts but if you are willing, you can achieve all sorts of cool results by tuning custom models.
Performance Metrics: Use the pre-defined success metrics (from Step 1) to evaluate the model's performance on the validation and test sets. Common metrics include:
Regression: Mean Squared Error (MSE), Root Mean Squared Error (RMSE), R-squared, Mean Absolute Error (MAE).
Classification: Accuracy, Precision, Recall, F1-score, Area Under the ROC Curve (AUC).
Deployment, Implementation, and Project Completion:
Integration: Plan how the model's results will inform design decisions or integrate with existing software. This could involve direct use of outputs, using insights to guide choices, or software integration.
Validation: Rigorously validate the model's performance on a held-out test set to ensure it meets pre-defined success criteria before implementation.
Efficiency: Confirm the model's computational speed is adequate for the project's timeframe. Consider simplification if needed.
Documentation: Thoroughly document the entire process: problem definition, data, model, validation results, limitations, and usage instructions.
Project Completion: Ensure clear and repeatable results. Typically, ML models in engineering serve as inputs to traditional deliverables (drawings, reports, calculations) rather than being standalone deliverables, although this may evolve as we see more interest and implementation. The hand-over should enable other engineers to reproduce and review the model and associated results.
The Decision: A Continuous Process
This framework is not a one-time checklist, but rather an iterative process, like many aspects of our profession.
Throughout the project, continuously reassess the problem, the data, and the model's performance. Be prepared to adjust your approach, or even abandon the ML approach entirely, if the evidence suggests it is not the optimal solution.
The goal is not to force-fit ML, but to use it judiciously and effectively. The ultimate measure of success is not the sophistication of the algorithm, but the tangible improvement it brings to your engineering output.
We should acknowledge the prevalent – and often justified – institutional resistance to machine learning, or more broadly the catch-all term ‘AI’.
This resistance frequently stems from its conflation with the broader, and sometimes overhyped, realm of Artificial Intelligence, particularly the uncritical reliance on large language models. The "garbage in, garbage out" principle applies with unwavering force, and outsourcing critical thinking to an LLM is, frankly, a complete abandonment of your professional responsibility.
However, to dismiss ML entirely based on these concerns would be equally shortsighted. Used correctly, these tools offer a pathway to unprecedented leverage, amplifying our ability to tackle complex engineering problems across a vast spectrum. A little knowledge is a dangerous thing, as any foray into the stock market will quickly confirm.
But informed knowledge, coupled with an understanding of engineering fundamentals, is the foundation for using machine learning effectively.
Proficiency with these tools, acquired now, is an investment in future adaptability. The sky is the limit for those who understand how to wield data effectively. While I, and many of you, have been beating this drum for a while, an entrenched resistance – and a healthy skepticism – persists within the engineering community, although I think the tide is changing.
To younger engineers, I offer this specific caution: While ML tools can undoubtedly enhance efficiency and effectiveness, they are not a substitute for rigorous engineering judgment.
One obvious mistake, one clear oversight due to over-reliance on an algorithm, can instantly erode the trust and credibility you've built. This is painfully evident in the careless use of LLMs, where a hastily pasted ChatGPT response into an email instantly betrays a lack of genuine thought. This can be hard to recover from!
The path forward is a tightrope.
The allure of the cutting edge, especially as technology hurtles forward, is undeniable. I struggle to balance enthusiasm with skepticism.
It's a constant process of sifting through the noise, identifying the truly useful tools, and integrating them thoughtfully into practice.
“To use a little, you have to know a lot.”
- Jean Jaques Machado
The goal is informed application, driven by engineering principles, not technological novelty. Don’t get suckered.
How are you integrating machine learning into your engineering practice? What tools/frameworks have worked for you?
See you in the next one.
James 🌊