The Evolution of RAG: Opportunities for Professional Engineers

Harnessing Retrieval Augment Generation (RAG) Models in Civil and Structural Engineering: Where We Are and Where We're Headed

Happy new year folks.

I hope you’re well rested and ready for whatever peculiarities 2025 will unleash upon us.

The world feels like it’s getting stranger each year, as if the lunatics are running the asylum. Fortunately for us, engineering as an industry feels relatively sane.

I’ve been meaning to provide a broader flocode update, what I’m up to and plans for this year so please bear with me on that front.

To those who take the time to read this newsletter: thank you. In an age of endless distractions and dubious certainties, your attention is an act of trust I don’t take lightly.

Today’s topic on RAG is one of deep personal interest so I hope you like this one. It forms the basis of a larger project I am currently working on. More on that soon.

James 🌊

---

Large language models, with their generative, nondeterministic nature, are great for creative tasks like brainstorming, marketing, and drafting text where absolute accuracy isn’t critical and mistakes are low-stakes.

But for professional engineers, precision and reliability are non-negotiable. This is why I’m cautious with these tools yet I’m determined to understand how they can be adapted with the right guardrails to meet the standards we need in engineering work.

Over the last couple of years, I’ve been exploring Retrieval-Augmented Generation (RAG) models across various platforms—OpenAI, Anthropic, Google, Meta and some custom systems I’ve built with LangChain and LlamaIndex. Llamaindex is the most effective for my purposes.

Each of these models has strengths, but also notable limitations, especially when working with the kinds of documents that civil and structural engineers frequently encounter. Complex engineering reports, which often cover intricate technical subjects with plots/figures/tabular data, can sometimes lead these models to generate confusing or inaccurate responses. This is partly because engineering documents require not only fact extraction but a broader understanding of context and industry-specific logic to ensure correct interpretation.

For those less familiar with the technical side of RAG, it’s essentially what powers the “document analysis” feature in many large language models (LLMs) today. When you upload a document—be it a PDF, Word file, or a text file—and ask the model questions specifically about its contents, you’re engaging with a basic RAG pipeline. While many of us may have used this function without knowing the term, you’ll recognize its core value: querying specific documents to get contextualized answers rather than relying on general web searches or memory.

A simplified model of a basic RAG application

However, while these models provide utility, they require thorough validation and oversight in fields like engineering, where accuracy and liability are the name of the game.

This article examines the current state of RAG models, their applications, and why recent advances in reasoning and logic might represent a turning point for those of us in technical fields. My hope is that these tools allow me to be more effective as an engineer, and so far, this has been the case.