#013 - Python Essentials | 01 - Jupyter Notebooks Part 1/3
A Concise Guide for Engineers on the Initial Steps in Utilizing Jupyter Notebooks, Covering Their Structure, Cell Types, and Environment Setup Essentials.
This article is part of the Flocode Python Essentials Series, intended to teach engineers from the ground up how they can get started with Python. Previous entries in the series include:
#010 - A Roadmap for Learning Python for Civil/Structural Engineering
#011 - Python Essentials | 00 - Programming Fundamentals
Introduction to Jupyter Notebooks for Engineers
Learning to use Jupyter Notebooks is one of the first steps in understanding and using Python as an engineer. While learning Python syntax is an obvious requirement, Jupyter Notebooks are crucial for creating shareable, well-documented work that captures not just code but also your ideas, methods and processes.
They serve as the canvas, while Python is the paintbrush.
A Jupyter Notebook is an open-source web application that allows you to create and share documents containing live code, equations, visualizations, and narrative text. Jupyter supports over 40 programming languages, including Python. We will focus on Python.
This post explores what Jupyter Notebooks are, their key features, and why they are worth your time as tools for practicing engineers engaged in technical work.
Jupyter notebooks allow you to write an interactive design document which can be updated and modified as you progress your design with everything in one place and by everything, this could mean:
Objectives
Project specific data
Methodology
Calculations
Results/Visualizations
Conclusions
References
You can connect any data structures, software or online resources (through an API), allowing your work to become as dynamic and responsive as you wish. For most engineers, this is a different approach to design.
To understand the real benefits and power of Jupyter Notebooks, you need to have some basic Python skills under your belt. We will cover those soon. For today, we’re just setting the stage.
Understanding Jupyter Notebooks
Jupyter Notebooks are popular in research and data science but have grown in popularity in other sectors due to their versatility and ease of use.
This has positioned them as potential alternatives or supplements to traditional calculation tools like Mathcad and Excel, which have been staples in engineering for years. I was an Excel user. Tears were shed in the arduous quest to hunt for cell references, find dependent cells, construct macros, and grapple with Excel’s VBA programming language. It’s tough sleddin’, as they say here in British Columbia.
Traditionally, in civil and structural engineering, you carry out your work, record your notes and observations in whatever tools or systems you choose, and once you’re satisfied, you write a design report summarizing your results, or you create a drawing that meets your design objective.
Jupyter provides an alternative path, allowing you to write an interactive design document that can be updated and modified as you progress your design. Seeing real-time updates or automation for calculations based on different software outputs is tremendously powerful and can extend your technical reach considerably.
But let’s start at the start.
The Architecture
There are two important things to know about Jupyter Notebooks so that you don’t hit the same speedbumps I did.
Basic Jupyter Architecture:
The front end: What you see and interact with, where you type your code, text and place your images. This is where you do all of your work. It’s pretty straightforward.
The kernel: This is your notebook's operating system or local environment. This runs behind the scenes. The kernel manages how your notebook functions, i.e., renders your plots, imports packages and modules, and executes your code. The default kernel is IPython, specifically for the Python language, but Jupyter kernels can also work with many other programming languages. The kernel is installed automatically as you’re setting up, but it’s important to remember because sometimes the kernel crashes or needs reactivation. If some other indescribable error occurs, it’s probably the kernel.
Apologies, I couldn’t not make this.
Jupyter Notebooks, JupyterLab, JupyterLab Desktop - what’s the difference?
It’s confusing, but they are all versions of the same thing, and the term Jupyter Notebook generally refers to the interactive notebook functionality.
Jupyter Notebooks are the original interactive notebook environment inside your browser—the most straightforward and rawest form of Project Jupyter.
JupyterLab is a notebook with some extra quality-of-life functionalities. It is a step up from the classic Jupyter Notebook. It’s pretty handy and runs in your web browser.
JupyterLab Desktop is a desktop application that provides a few extra bells and whistles to ease into Jupyter. This option is great if you want a local setup with minimal headaches.
Where do I begin?
I suggest starting with a cloud notebook with any providers listed below. This removes many initial stumbling blocks and allows you to focus on learning to use Jupyter Notebooks rather than getting confused about the environment setup (another can of worms for another day).
Offers free access to GPUs and TPUs for faster computation. Integrated with Google Drive for easy file sharing and storage. User-friendly interface for those familiar with Google products.
Provides a vast array of datasets and community-shared notebooks. Ideal for data science projects and machine learning. Enables participation in competitions with data scientists worldwide. Kaggle is a fantastic community and resource for data science.
Integrates directly with GitHub repositories for seamless code management. Offers a configurable, cloud-based development environment. Ideal for collaborative projects and version control.
You can check out this free codespace on GitHub.
Flocode's Introductory Codespace for Engineers 🚀
Feel free to mess around and experiment. If you don’t know any Python syntax, skip this for now or try it anyway; a few examples of simple syntax are included. I will cover Python syntax in an upcoming article.
Once you get a general idea of how it works, you should consider installing JupyterLab or a preferred interactive development environment (IDE) like Visual Studio Code (VS Code). Working locally gives you more control over your environment.
There are many spirited debates online on the best IDE. I won’t enter that arena. I like VS code, so I use it. If you feel strongly about this, unload in the comments.
VS Code has many helpful features to help your productivity, so I recommend you start sooner rather than later. Pycharm, Sublime and Spyder are other popular IDE options.
Sidenote: I use Visual Studio Code as my primary interactive development environment (IDE). Jupyter can be installed as an extension for Jupyter Notebooks with extra functionality and integrations with other systems.
Interactive Coding, Visualization, and Documentation
The user interface of Jupyter is pretty simple, and there’s excellent information provided in the official Jupyter Docs if you want to dig deeper. Essentially, you have two types of cells in your notebook.
Markdown Cells
Codes Cells
Markdown Cells
In Jupyter Notebooks, you can seamlessly blend descriptive text with code, creating a document that can perform computations and explain them in detail. This is known as "literate programming," facilitated through Markdown cells.
Markdown is a lightweight markup language that allows you to add formatting elements to plain text. It's used within Jupyter (specifically in IPython) to create rich text documentation alongside your code. You can format your text using various Markdown syntax elements when writing in a Markdown cell. For instance:
Emphasis: You can use asterisks (*) or underscores (_) to italicize words (e.g., *italic* or _italic_) and double asterisks or underscores for bold text (e.g., **bold** or __bold__).
Lists: You can create bulleted or numbered lists to organize information.
Headings: Headings are created with hash signs (#). You can use one to six hash signs to create headings of different levels (e.g.,
# Heading 1for a top-level heading,## Heading 2for a second-level heading, and so on). These headings structure your document and become clickable links within the notebook, making navigation easier.Links and Images: Markdown allows you to insert hyperlinks and images, enhancing the interactivity of your document.
Once you get the hang of the markdown shortcuts, writing clean, descriptive content is ten times easier. All of my articles are written in markdown (using Notion).
When you run a Markdown cell, the Markdown code is transformed (rendered) into formatted text. This conversion makes your text more readable and visually appealing.
Markdown in Jupyter Notebooks supports arbitrary HTML code, offering even more flexibility for complex formatting needs. This feature is handy when exporting your notebook to different formats, like PDFs, as it ensures that the rich text maintains its structure and formatting in the exported document.
Code Cells
A code cell in a Jupyter Notebook is a dynamic element where you can write and edit code. This code cell features syntax highlighting and tab completion for better readability and ease of coding. The type of programming language you can use in a code cell depends on the notebook's kernel. The default kernel in Jupyter is IPython, designed for executing Python code.
When you run or "execute" a code cell, the code within it is sent to the notebook's kernel. The kernel processes this code and returns the results displayed directly below the code cell as its output. Importantly, the output from a code cell isn't limited to just text.
It can include a variety of rich media formats, such as graphical figures created with Python libraries like Matplotlib or HTML tables often seen in data analysis with Pandas. This feature is part of what's known as IPython's "rich display capability," allowing for a more interactive and visually engaging way to present computation results within the notebook.
This is a major advantage because you can output all sorts of visualizations using different Python libraries. Some common use cases for me, among many more, include;
Financial Modelling
Statistical Distributions
Analytics and Data Processing
Diagrammatic Sketches
3D and Volumetric Data
Structural Analysis
Hydraulic Modelling
Maps/Photo Models
Tables, Charts, and Plots of all types (see Matplotlib or Seaborn)
Unlike traditional scripting environments, Jupyter Notebooks provide immediate feedback by allowing code execution in isolated cells. This feature is particularly beneficial for trial-and-error coding and iterative data analysis.
It’s easy to see how useful this is for engineering. You can isolate specific segments of your code; perhaps you want one code cell to calculate soil-bearing capacities or spring stiffness properties and a new separate code cell to calculate your foundation system's structural dimensions or capacities. You can isolate and confine specific topics to individual code blocks.
This is helpful for quality control and organization.
Closing
In this initial overview of Jupyter Notebooks, we've laid the essential groundwork for our upcoming deep dives into Python. This preparation is crucial; proper tool selection and setup can save significant time and effort later.
“Give me six hours to chop down a tree, and I will spend the first four sharpening the axe.” - Abraham Lincoln, allegedly.
In reflecting on my own experiences starting out, I think I jumped the gun on Jupyter Notebooks and dove in before I understood them. This led to unnecessary complications (i.e. cursing under my breath at my screen), especially with environment setups and library compatibility.
These issues, though seemingly minor at the outset, are critical for smooth progress, building good habits and minimizing frustration. We will cover these topics in depth in future articles. For now, understanding the basics is vital.
Today's focus was to help you understand the structure of Jupyter Notebooks.
In upcoming Python Essentials articles, we’ll explore more features of Jupyter Notebooks and start breaking down Python syntax. If you encounter initial hurdles, particularly related to environment and package management, recognize it as a standard part of the learning curve. You will prevail; don’t sweat the small stuff.
The goal here is to learn how to code and approach it with efficiency and a pragmatic mindset.
Please share the flocode newsletter with colleagues who may find value in it. We’re just getting started.
Stay tuned.
James 🌊
One problem I’ve had with using Jupyter for calcs is that the output pdf (using nbconvert) does not look like an excel sheet “report” (mainly the lack of header and borders). So I always get comments about my calcs not being aesthetically consistent with others. I’ve been able to add a header using an html table in a markdown cell, any ideas for borders? Or other template features to make the output more familiar to reviewers?