Embrace the Chaos

Introduction to Chaos Theory

“The flap of a butterfly’s wings in Brazil can lead to a hurricane in Texas.”

While not meant to be taken literally, this metaphor captures the essence of Chaos Theory: small changes in a system's initial conditions can lead to large and unpredictable outcomes. This idea, often called the butterfly effect, lies at the heart of chaos theory and highlights the sensitive dependence on initial conditions seen in many natural and mathematical systems.

Chaos Theory states that "within the apparent randomness of chaotic, complex systems, there are underlying patterns, interconnectedness, constant feedback loops, repetition, self-similarity, fractals, and self-organization."^[1] In simple terms, even though these systems may appear unpredictable, they follow deeper rules. Small changes in initial conditions can lead to drastically different outcomes as time progresses, making long-term prediction extremely difficult.

One common example of a chaotic system is a double pendulum. A double pendulum’s position can be found using deterministic equations. When given a set of initial values, we can predict where it will be at any given time. So why is it an example of a chaotic system? A double pendulum is an example of deterministic chaos. We can consistently calculate its position if exact data is available, but that can be hard to come by.

The position of the double pendulum is extremely sensitive, and a change in significant figures can completely change its trajectory. This can be shown in the double pendulum at the top of the article. Ten pendulums are displayed, with the only difference being that each theta value is increased by a tiny amount. Within a few seconds, all the pendulums break off into completely different pathways. A quote that describes this well comes from Edward Lorenz, a mathematician who laid the foundations of chaos theory:

Chaos: When the present determines the future but the approximate present does not approximately determine the future.

The History of Chaos Theory

The idea of Chaos Theory stems from the previously mentioned mathematician and meteorologist Edward Lorenz in the 1960s. Throughout his work as a meteorologist, he concluded that weather patterns are nonlinear and can be volatile. This was enforced when one day he decided to go back and recreate a past weather sequence. When he initially made the calculations a year prior, he used six significant figures, but when running through them again he used three. One would expect this decimal change to be insignificant, but that was far from the truth. The two weather sequences were vastly different, showing that seemingly negligible factors can completely change outcomes.^[2]

The Lorenz Attractor

Lorenz is well known for the famous Lorenz Attractor, a model often associated with the Butterfly Effect. This model was developed by Lorenz with the help of Ellen Fetter and Margaret Hamilton.^[3] These equations show a simplified model of atmospheric convection, and what makes this set of equations notable is its extreme sensitivity to initial conditions.

The three differential equations are:

The purpose of these equations is to "relate the properties of a two-dimensional fluid layer uniformly warmed from below and cooled from above."^[3] x is proportional to the rate of convection, y is proportional to the horizontal temperature variation, and z is proportional to the vertical temperature variation.^[3]

The values σ sigma, ρ rho, and β beta are constants that can be changed, but picking random numbers will likely result in the particles flying towards infinity. The most common constants, picked by Lorenz himself, are:

Starting from an arbitrary point, future points are calculated based on the previous. It is important to note that not all inputs in this equation result in a chaotic output. For example, having initial values (0,0,0) will result in all derivatives equaling 0. The point will forever stay at the origin. On the flip side, inputting large values will result in the particle shooting towards infinity. The most interesting behavior happens when we pick starting points in a certain "sweet spot."

As part of my research, I programmed a 3D model of the Lorenz Attractor, viewable through the button above. I initialized the point at (0.01, 0, 0) to ensure non-zero derivatives and a true chaotic attractor. What’s fascinating about this model is that the trajectory never exactly retraces its steps. If it did, the motion would no longer be chaotic; it would become periodic or predictable. Even more interestingly, if we start 100 points close together, some will diverge to infinity, and some will stay trapped in the attractor’s general shape. Even though they follow similar patterns, no two paths are identical. A tiny difference in starting points leads to drastically different futures.^[4]

Math Behind the Double Pendulum^[5]

The orientation of a double pendulum can be shown in the diagram. When doing calculations, we have several variables we will be working with:

We can also state that the location of the first ball is (x₁, y₁) and the location of the second ball is (x₂, y₂). The point the pendulum is hanging from is (x₀, y₀). We are also going to assume that x₀, y₀ = 0. Using basic trigonometry, we are able to write these points as the following:

Now we need the derivatives of each coordinate. A derivative is denoted with a dot:

To calculate the change in the θ values, we are going to use a Lagrangian. The Lagrangian describes a system in terms of its kinetic and potential energy and is founded on the principle of least action. The formula for the Lagrangian is written as:

In other words, the Lagrangian equals the kinetic energy T minus the potential energy V within the system. This means the next step is to solve for T and V:

Simplifying with trigonometric identities gives:

This can be rewritten as:

Now we substitute this information into the Lagrangian:

We will now apply the Euler–Lagrange equations with respect to θ₁ and θ₂. The formula states that the total time derivative of the Lagrangian’s partial derivative with respect to velocity, minus its partial derivative with respect to position, must equal zero. This allows us to get the angular acceleration for both theta values.

The last step is to get the explicit equations for θ̈₁ and θ̈₂ using the Euler-Lagrange equations.

Where Δθ = θ₁ − θ₂.

These are now all the equations used for the program, with the primary equations being the Euler-Lagrange equations for angular acceleration.

My Program

This project simulates the motion of a double pendulum using JavaScript and an HTML canvas. The program is built around the physical model of a double pendulum: two rods connected end-to-end, swinging under the influence of gravity. To calculate the motion, I used the standard equations for a double pendulum's angular accelerations.

Input Handling

The program uses sliders to collect user input for:

Pendulum Class

These values are passed into a Pendulum class, which initializes the angular positions:

For each new Pendulum object created, θ₁ is slightly offset to visualize how small differences in starting conditions can lead to dramatically different motion — a feature of chaotic systems.

Calculating Motion

Inside the class, I calculate the angular acceleration based on the current positions and velocities. Then, at every frame:

Visualization

Using an HTML canvas, I draw lines connecting:

This real-time drawing creates a smooth animation of the double pendulum swinging chaotically over time. We are able to see the trail of the pendulum by changing the opacity of the background, which results in more frames being needed to cover up previous drawings.

Note: A double pendulum experiences air resistance and friction at the pivots, resulting in imperfect energy transfer and gradual energy loss over time. This program models the idealized case, focusing solely on the pendulum’s motion without accounting for these dissipative effects.

Applications of Chaos Theory

The findings in Chaos Theory have many real-world applications. Many of the systems we use in our day-to-day lives are chaotic, unpredictable, and reliant on initial conditions. Examples include weather patterns, economics, and artificial intelligence. The goal of using Chaos Theory in these fields is to find order in messy systems.

Weather Prediction

As the Lorenz Attractor demonstrates, weather patterns are extremely sensitive to changes in initial conditions, and small errors can snowball into vastly different weather outcomes. This sensitivity has shaped how meteorologists predict weather today. Forecasts have a limited window of accuracy, typically about 10 days because small measurement errors compound over time. Beyond that window, predictions become increasingly unreliable, and new calculations based on updated measurements must be made. For long-term weather forecasting, meteorologists rely more on statistical models based on historical data. For example, by analyzing temperature averages from the past 10 years, we can estimate what conditions might be like six months from now. However, such predictions are broad trends rather than precise forecasts, and they are refined as the specific date approaches and more current data becomes available.^[6]

Machine Learning Models

Although Chaos Theory connects to a vast number of fields, the last one I want to explore is Machine Learning. When reading about Chaos Theory, I was surprised to learn that a machine learning model could even fall under its umbrella. While these models are not chaotic in the classical sense, since they are not fully deterministic, the principles of Chaos Theory still have relevance in how we develop and train them. Large machine learning models can exhibit sensitivities to their initial weights or input prompts, and even small alterations may lead to drastically different outputs.^[7]

While exploring this idea, I came across a visual representation of a loss landscape, which maps a model’s error during training: peaks represent high error, and valleys represent low error. The training process involves navigating this landscape using gradient descent, seeking paths that lead to the valleys of minimal error. However, even slight variations in input data or initialization can steer the model along a different trajectory, potentially causing it to settle in a suboptimal local minimum. This process bears a resemblance to chaotic systems, where small shifts can dramatically alter outcomes.

Although machine learning models are not typically considered deterministic chaotic systems, the concepts from Chaos Theory offer valuable insights into their training dynamics. Being mindful of overly sensitive models, and recognizing how minor changes in data or initialization can impact training, is essential for building robust and generalizable systems.

Conclusion / Reflection

Chaos Theory is deeply tied to many complex systems we rely on daily. It not only helps us predict and understand chaotic behavior, but also plays a crucial role in preventing the development of unstable models, such as certain machine learning systems. Through the two projects I completed, I gained valuable insight into the fascinating mathematics underlying these models. I especially enjoyed observing how a tiny change in the input could lead to a drastically different output. This project has significant potential for further exploration; I’m particularly interested in diving into topics like Arnold's Cat Map next.