Dynamical Systems Theory: What in the World is it?

Mike Hochman

This page is meant to explain in a very general and non-technical way a few things about my field of research. It's meant for freshmen students, my friends, my mom (hi mom!) and anyone else who is interested; you don't have to be a mathematician to read it. I've tried not to lie too much, though sometimes I've bent the truth a little in the interest of clarity (I hope, anyway). If you would like more details take a look at the list of references at the end. If you have any questions or suggestions I'd be glad to hear from you.

What is dynamics about?

Dynamical systems theory attempts to understand, or at least describe, the changes over time that occur in physical and artificial "systems". Examples of such systems include:

The solar system (sun and planets),
The weather,
The motion of billiard balls on a billiard table,
Sugar dissolving in a cup of coffee,
The growth of crystals
The stock market,
The formation of traffic jams,
The behavior of the decimal digits of the square root of 2;

and so on.

Many areas of biology, physics, economics and applied mathematics involve a detailed analysis of systems like these, based on the particular laws govorning the way they change (which, in turn, are derived from a suitable theory -- Newtonian mechanics, mathematical economics, etc.). This is often a case-by-case effort, with each field and subfield using its own techniques and tricks (not to mention jargon).

The mathematical description of dynamical systems

All these models can be unified conceptually in the mathematical notion of a dynamical system, which consists of two parts: the phase space and the dynamics.

The phase space of a dynamical system is the collection of all possible world-states for the system in question. Each world state represents a complete snapshot of the system at some moment in time.

The dynamics is a rule that, given point (state) in the phase space representing the current state of the world, determines another point representing the state that the world will be in after the passage of one unit of time. In mathematical language, the dynamics is a function mapping world states into world states.

For example, if we are studying planetary motion then a world-state might consist of information regarding the location and velocities of all planets and stars in some neighborhood of the solar system (or the galaxy, if we are ambitious); and the dynamics would be derived from the usual laws of gravity that tell us what the forces are between bodies given their masses and distance apart.

Once an initial world-state is chosen, the dynamics determines the world-state at all future times: if you want to know what the current state turns into after two units of time just apply the rule once, giving the next state, and apply the rule again to that state to get the one after it. Applying the rule again and again we obtain all future states (in some cases we can also reverse this and get past states).

Notice that by describing the system in this we we have made a few assumtions about the way the system changes. First, we assume that the current state of the world uniquely determines the next state, and second, that the transition to the "next" state is independent of the time at which the transition occurs. In other words, if we perform an experiment by placing the system in some state and letting it evolve for one or several time steps, the result will be the same whether we do the experiment today or tomorrow or next week. There can be systems that don't satisfy these two assumptions, but I will only talk about those that do.

Abstraction

Abstract dynamics is the study of dynamical systems based only on the description explained above, without any additional information about the origin of the dynamics. In particular we do not assume that the world states have any interpretation except as points in the phase space (although in many cases such an interpretation is available), and instead we think of them as abstract "points" in a "space".

Similarly, we usually do not assume that the rule has any particular form. For example, we do not require that it can be written using arithmetic operations (this would only make sense, in any case, if the world states were composed of numbers!); or then it can be computed efficiently, or even that it can be written down in some concrete and finite form. We only assume that the rule exists; and we would like to draw conclusions from that.

This point of view is actually too restrictive, so actually we retain a little more information, though still much less than in concrete models. For example, we often retain some information about which states are close to each other (this is called "topological dynamics"). In the example of plantary motion, two world states might be thought of as being close to each other if corresponding astronomical objects in them have similar positions and velocities. In the abstract version of this model we are therefore left with world-states at varying degrees of closeness to each other, but no longer know how to decompose a world-state into information about planets and stars.

There are other kinds of information about the phase space and dynamics that is sometimes retained, such as the relative probabilities of different world-states (this gives rise to "ergodic theory"), or certain geometric information (giving rise to "smooth dynamics").

What is abstraction good for?

Passing from a concrete model to its abstract representation obvously entails some loss of information, and having given up so much detail about the system we can't expect to get results which are more precise than an analysis of the original model would have given, or to calculate things more efficiently (after all, in order to predict the weather you do need to know something about weather!). The abstract theory is not meant to replace detailed analysis.

Nonetheless, there are good reasons to study the abstract model. One is precisely its generality: any technique or concept we can deduce in the abstract model will immediately apply to any concrete model, giving tools that can be used on the entire spectrum of dynamical systems.

Stripping away the details of a system also can focus attention on more essential properties of it. It is often a crucial step if we want to compare or draw analogies between systems of different kinds (apples and oranges, so to speak): often, similarities are only apparent when you step back and look at the big picture. For example, some models of the stock market and of the motion of billiard balls are very closely related, but you can't see this until you describe them abstractly and forget any interpretation of the variables as prices or velocities.

There is also another advantage to the abstract model. It is often the case that the details are actually irrelevant to the problem, and adopting the abstract point of view can lead to new insights that would otherwise be obscured. This is a psychological gain more than a mathematical one, but it can be quite important.

Here's a simple example from high-school math that may shed some light on this. Consider those "real world" problems that you often see, eg: train A leaves New York at 100mph, train B lease Boston at 20mph, how far from Boston will they meet if the distance is 400 miles? In high school we learn to represent such a problem with equations. The point is that once you write down the equation, you can safely forget that "x" represents the distance traveled by train A; instead "x" is an abstract variable and you can focus on solving the equation. Morover, the methods for solving the equation are not special to train problems; they work for car problems, horse problems, etc. What the equation represents is totally irrelevant for this process. In the case of dynamics we are losing a lot more information than just the names of things, but it turns out enough is left that interesting conclusions can be drawn.

Classification of dynamical systems: philosophy and examples

A very general problem in abstract dynamics is to understand when two systems are "the same", either precisely or in some fuzzier sense. The following examples will give some idea of what this means:

The dynamics of dissolving a drop of red ink in a cup of water is essentially "the same" as the process of a drop of black ink.
In the ink example, if we run time backwards we get a different dynamics than running it forward, because when time goes forward the ink becomes more and more spread out and dissolved, whereas if time goes backward it starts out dissolved and becomes more and more concentrated in a drop (this observation is the essence of the second law of thermodynamics
The dynamics of billiard balls moving on a frictionless table look the same if we run them forward or backward (if I showed you a movie, you couldn't deduce me which way time was going). Therefore the dynamics of billiard balls is essentially different from the dynamics of dissolving ink (actually, I am lying here. Ink and water are made up of atoms that bounce around a lot like billiard balls. more on this later).

One of the great successes of the abstract theory has been to show that many apparently different systems are in fact the same. However, it turns out that classifying all dynamical systems individually is too hard a problem, because there simply are too many of them. Instead, we can try to classify them according to coarser measures of similarity and dissimilarity of their dynamics.

Example: stationary vs. non-stationary dynamics

One distinction that can be made is between systems which are in a "stable state", called stationary systems, and those which are not. Being stationary does not mean that there is no longer any change in the system; rather, it means that "the more things change the more they stay the same". A more precise definition of stationarity is that in a stationary system, if we observe the system at two times, we cannot, based on the observation, know for sure which of the two times came earlier.

For example, if we I show you two short videos of billiards balls rolling around on a frictionless table, you will not be able to say with any certainty which picture was taken first. Hence this system is stationary. On the other hand, if there is friction, then we are in the non-stationary situation, because the balls will slow down as time progresses, and their speed gives us some information about when the observation was made (but from the moment the balls stop completely, we are again in the stationary case, since no further change occurs. This is a common feature of physical systems -- even when they are not stationary, they often approach a stationary phase).

Often a system will begin in a non-stationary state and evolve towards a stationary one, like the billiards with friction. That this should occur is not at all obvious. It is an interesting result of the abstract theory that (under some weak assumptions) there always axist stationary states, no matter what the system.

For another example let's go back to ink dissolving in water. This process doesn't seem to be stationary but as time goes on it approaches those states where the ink is totally dissolved and evenly distributed throughout the water; these "totally dissolved" states form a stationary system, because once the ink is totally dissolved, it stays that way (I am lying a little here, but a full explanation would get messy. I'd like to mention, though, that the problem of understanding whether this system is stationary or not was a major conundrum in the late 19th century and confounded the greatest minds of that era).

Example: chaotict vs. non-chaotic (deterministic) dynamics

Another important distinction is between "chaotic" and "non-chaotic" dynamics, that is, between systems which exhibit sufficient "randomness" and "unpredictability", as opposed to those which do not. Examples of chaotic systems include a great many physical systems (e.g. the weather) as well as social and economic systems (e.g. the stock market). A word of caution: The notion of chaos means different things to different people and is not a well-defined mathematical concept. Among its interpretations there are also a lot of populistic and misleading ones (some of which have little to do with science or math).

You may have noticed that I described dynamics as a rule govorning the evolution of a system, but also spoke of "randomness". This might seem contradictory: if every state uniquely determines the next state, where does the randomness come from? This is resolved if we admit that in the real world, we usually do not know the state of the world precisely. For example, we can determine a lot about todays weather by measuring temperature and pressure at a bunch of locations. But this does not give us complete information: we are very, very far from knowing the location and velocity of every molecule in the atmosphere (this is undoubtedly impossible). If we had complete information we could predict tomorrows weather according to our physical theories of atomic motion and interactions. But the uncertainty that we have about the present state of the world means that our prediction of tomorrow's world-state will also be uncertain, and the uncertainty will often grow the further into the future we try to predict.

On the other hand, in some systems the uncertainty does not increase. Roughly speaking, such systems are called deterministic. It is an interesting and non-trivial fact that if the uncertainty does not increase then it must equal zero! in other words, either our ability to predict the future based on incomplete information decays very rapidly as we look farther and farther into the future, or else given enough information we can make a completely accurate prediction about the future, for all times! This dichotomy is only of partial practical use, since to make perfect prediction you need more information than is practically available, but it has interesting theoretical (and philosophical) implications.

Example: invariants

An invariant of a dynamical system is some quantity you can measure (either in practice or in theory) which comes out the same when you compute it for different systems that are "the same". For example, the property of containing billiard balls is not an invariant, because a system with billiard balls is the same as a system with ping pong balls (after some minor adjustments). Actually, in the abstract model there is no meaning to the question "does the system contain billiard balls?", because in the abstract model we have forgotten all about what the states mean.

Returning to the discussion of uncertainty, it turns out that it's possible to measure the rate at which the amount of uncertainty grows as you predict farther and farther into the future; and this quantity -- the rate of uncertainty growth -- is an invariant. This invariant is called entropy; it is related to but no the same as the notion of entropy in thermodynamics. One common interpretation of the word "chaos" is as "positive entropy"; i.e., a sustem is chaotic if the farther you look into the future, the poorer your eyesight becomes. Entropy gives even more information, since it gives a basis for claiming that one system is more "chaiotic" than another, in the sense that our eyesight deteriorates more rapidly in one than in the other.

There are not so many other known invariants (at least, useful ones). One other example is the periodicity of the system, i.e. at what regular sequence of times does it return close to its initial state. The weather, for example, is approximately periodic with period 365 days (1 solar year). On the other hand billiard balls on frictionless table are typically not periodic.

Some basic questions

Let's go back for a moment to our goal of understanding the similarities and differences between different systems. Once we have identified some useful properties or invariants of dynamical systems, there are many questions we can ask. Here are a few of my favorites:

What are the relationships between the different attributes? For example, enough periodicity turns out to imply zero entropy (this shouldn't be surprising. If a system is periodic it means the past repeats regularly, so knowing the past we can predict the future perfectly; and we said this is the same as zero entropy).

When we are observing an unknown system, can we determin its attributes by observing a sequence of measurements? Notice that there is a difference between figuring them out from knowledge of the system and figuring them out by "participating" in the system. For example, if you know the law of gravity you can try to compute things about the solar sustem. But this is a different problem from that of an ancient astronomer who is ignorant of gravity but has at his disposal observations and measurements of the heavans. It turns out that even when we do not know the rules, we can sometimes estimate the entropy of a system, though not necessarily its periodicity!

Given some description of the rule govorning the dynamics of a system, can we use it to decide if the system has some attribute or not? Surprisingly, this is sometimes impossible! there are very simple rules that can generate dynamics so complicated that we can actually prove that we cannot understand them completely!.

Do "most" systems have a certain attribute? This is a somewhat subtle question because it depends what you mean by "most", and on some other details of the way you represent the syste, But it turns out that, rather generally, either most do or most do (you often can't have "half and half"). For example, most systems have zero entropy, but most systems are also aperiodic.

Which real-world systems have which attributes? As we already mentioned, there are many real-world system that display chaotic behavior like positive entropy, but there is a long list about which we are still not sure, and it is a challenge to develope methods for deciding this. And so on.

There are also many interesting applications of dynamical systems theory to other areas of mathematics, particularly combinatorics and number theory, but I won't go into these.