Wiener Process

Key Points:

• A Wiener process is any real-valued, continuous-time-stochastic process that itself varies continuously. It was named in honor of Norbert Wiener.
• The Wiener process is used to represent the integral of a white noise Gaussian process and is useful as a model of noise in electronics engineering.
• Wiener processes are used in everything from electronics engineering to derivatives trading. Some financial models outright assume that price movements in markets are Brownian, making Wiener processes the natural means by which to examine them and plan a trading strategy.

What is a Wiener Process?

To most, the idea of studying randomness seems like a contradiction in terms. After all, randomness is chaotic, the very antithesis of the lawlike behavior that something must exhibit to make it amenable to any sort of explanation at all. In a quest to understand randomness, where would one even begin?

And yet, the world seems pervaded by randomness which cries out for explanation — not only the natural world that was here before we were but also the technological world that we humans have built with our own hands. Observe how grains of pollen move through still water or how dust motes float upon the air in a room. The water is tranquil. No force that is discernible to us propels the pollen around — and yet, it moves. Still, less is it clear how or why it moves in just the way that it does, in its restless and bumbling drunken walk, anarchic yet oddly mesmerizing.

Though computers are presented as the very model of cool and undisturbed rationality, precisely the same sort of helter-skelter lawlessness pervades the computer’s very brain — the semiconductor. The semiconductor is an electronic circuit, and so, electrons stream through it constantly whenever it is in use. As it is subjected to progressively more strenuous use, its crystal heats up, causing electrons to jump from its valence band — a place of low energy — to its conduction band — a place of high energy. When electrons in a semiconductor leave a certain place behind and move on to another, the place they leave behind is called a hole.

Unbelievable as it may sound, the movement and distribution of holes in a semiconductor over time — just the inverse of the movement of the electrons themselves — follows the same sort of tumultuous pattern as that of the pollen moving through the water. It all looks like a topsy-turvy jumble — and yet, it is incumbent upon us to understand it. Given the central place of computers in our world, if we don’t, then we might as well kiss our marvelous technological civilization goodbye.

This ostensibly chaotic and rampageous behavior has a name. It is called Brownian motion, and when observed, it seems like something that sits on the very knife-edge of chaos — certainly not rigid or orderly, but also not quite wild enough to be utterly beyond comprehension. For a long time, this phenomenon was an enigma. Gradually, however, it came to be understood — and understood, paradoxical as this may sound, in an extremely precise way.

A great mathematical tool has allowed us this understanding and enabled us to describe and analyze Brownian motion in all of its varied manifestations. Given to the world by the American mathematician, philosopher, and founder of cybernetics Norbert Wiener, it is called, naturally enough, the Wiener process. We will therefore endeavor to explain, in the clearest way possible, how this critical part of the riddle of randomness was cracked.

Before we begin, however, a special caveat to all readers: Since Wiener processes are a dense and technical subject, any attempt to seriously explain what they are, how they work, and why knowing about them is useful will inevitably have to include some precise formal statements written in mathematical notation. For the benefit of those not versed in pure mathematics, we will attempt to explain every piece of mathematical notation and define every technical term, only employing as much mathematical symbolism throughout as is necessary to communicate meaningful understanding. Those interested in seeing the formal proofs of the theorems and other statements referred to here will have links that they can follow to satisfy their curiosity.

We encourage readers who may be intimidated by mathematical symbolism to persist, for Wiener processes are important for understanding crucial topics in fields as diverse as quantum physics, cosmology, finance, and electronic engineering. Knowing about Wiener processes will open a great many doors to understanding in these other fields as well.

Wiener Processes: An Exact Definition

A Wiener process is any real-valued, continuous-time stochastic process that itself varies continuously. To give its formal definition, all Wiener processes W_t have the following properties:

• W₀ = 0
• For all t > 0, all future increments W_t+Δt – W_t, with Δ > 0, are independent of all past values of the process W_s, where s ≤ t
• W_t+Δt – Wt ~ N(0, v)
• W_t is continuous in t

That is, the initial value of all Wiener processes is 0; past values of these processes do not influence any future changes in their value (this is what makes the processes stochastic); all of the process’ increments are Gaussian, which means that they are normally distributed, have a mean of 0 and a variance of v; and the process occurs in continuous time.

How do Wiener Processes Work?

A Discussion of Concepts

To clearly explain what a Wiener process is, it is first necessary to go back to basics and explain a few simpler concepts and how they are used mathematically. First, a process is any series of events starting at some time 0 and ending at some time t which starts at some initial value and ends at some final value. Time may move either discretely — that is, it may move in jumps from t₁ to t₂ with nothing in between — or it may move continuously — that is, along a perfectly smooth gradient. In continuous processes, the symbol Δ is used to refer to arbitrarily small changes either in time or in the value of the process. Thus, for example, the symbol t+Δt refers to the continuous movement of time past the point t. Previous events in the process may or may not influence how future events turn out.

The next important idea to understand is that of a stochastic process. Put simply, a stochastic process is any process whose subsequent states are not influenced by its past states. The probability that any given future state will occur does not vary concerning whatever the past states of the process happen to have been. The future states simply are what they are, and they are to be treated in isolation from past states.

Thus, for example, picking marbles of various colors out from a jar, placing them outside of the jar, and then choosing again is not a stochastic process, because whenever you pull out a marble of some given color, you slightly decrease the probability that you will pull out a marble of the same color the next time and slightly increase the probability that you will pull out a marble of a different color. However, if you were to place each marble that you chose back into the jar before each subsequent choice, the result would be a stochastic process.

Colloquially, a stochastic process is just a random process. This concept has an enormous breadth of different applications which reach into everything from the study of the growth of bacterial populations to the analysis of the flow of electric current as electronic circuitry happens to heat up.

The next major concept to absorb on the path to understanding Wiener processes is that of the Gaussian process. To understand what a Gaussian process is in applied mathematics, it is first necessary to explain a few subsidiary concepts. The most important of these is that of the normal distribution. The normal distribution is an expansive and complex topic in probability theory with countless applications to a myriad of areas in science and technology, and we need not discuss it in its full detail here. For our purposes, it will be enough to note that a continuous variable x is said to be normally distributed if plotting the probability distribution of its values creates a bell curve shape. The formula for the normal distribution is given by:

f(x) = (1/σ√2π)e^{-1/2[(x-μ)/σ]2}.

e and π symbolize the two numbers that they usually symbolize in pure mathematics, σ is the standard deviation of the distribution, and μ is the mean expected value of the distribution. To keep things as intuitive as possible, think of the standard deviation as a measure of the degree of dispersion of your variable away from the mean value. The term variance also refers to the square of the standard deviation.

Thus, for this article, when we refer to a normal distribution, we will use the notation N(μ, v), where μ is the mean and v is the variance. Thus, G_t ~ N(μ, v) will mean that some variable G_t is normally distributed with a mean of μ and a variance of v.

Next, we must explain what a linear combination is. Suppose that you have some set of values (G₁, G₂,… G_n). Suppose that you also have some set of constants, which can be either real or complex numbers, (a₁, a₂,… a_n). Thus, a linear combination of (G₁, G₂,… G_n) and (a₁, a₂,… a_n) would be (a₁G₁+a₂G₂+…a_nG_n). You can also combine any subsets of these two sets into smaller linear combinations.

Bringing everything together, we can define a Gaussian process as any stochastic process (G₁, G₂,… G_n) in which the values of every finite, real-valued linear combination of (G₁, G₂,… G_n) are themselves normally distributed. This notion of the Gaussian process in applied mathematics is not only used concerning processes that happen in one dimension but can be extended up to an arbitrary number of dimensions.

The final major concept that we must define is that of a Levy process. Levy processes, named after the French mathematician Paul Lévy, are stochastic processes with the following additional properties:

• L₀ = 0
• For all times 0≤t₁2<…t_n<∞, all increments of the process L_t2-L_t1, L_t3-L_t2,…L_tn-L_tn-1 are independent of one another
• For any st-L_s have the same probability distribution as L_t-s
• For any ε>0 and any t≥0, the limit as h goes to 0 of the probability of (|L_t+h-L_t|>ε) is 0

The first property means that all Levy processes start at an initial value of 0. The second property means that the value of each increment of a Levy process at any given point in time does not affect the value of any of the other increments at any other point in time.

The third property is usually described by saying that Levy processes have stationary increments, which roughly means that changes in their value depend only on the period taken up by each observation, but not on the time when that observation was started. The basic idea behind the fourth condition is that whenever the set that indexes the time changes in a Levy process converges on some value, then the probability distributions of that process also converge on that value — that is, as time gets closer and closer to moving in a smooth gradient throughout the process, then the process’ probability distributions also get closer and closer to changing in a smooth gradient.

The rather thorough dive into mathematical ideas that we’ve just conducted, though asking a great deal of the reader’s attention, has two essential advantages. First, it should have completely demystified the mathematical formulas and notation given in the above definition of a Wiener process. The foregoing discussion should have provided you with a relatively precise understanding of just what a Wiener process is. When we turn our focus to its real-world applications, this will make possible a deeper recognition and appreciation of the concept’s practical utility.

Secondly, now that we’ve defined all of the foregoing concepts, we can see the Weiner process from an angle that is not immediately obvious if all one does is scrutinize its formal definition. Once appreciated, this new angle communicates deeper insight. And that new angle is this: that Wiener processes can be thought of as anything with is both a Gaussian process and a Levy process. It is a special type of continuous-time stochastic process which has the properties of both.

The second point in the original definition hints that a Wiener process is also a Levy process; and the third part of the definition best reveals its Gaussian characteristics.

The Origins of the Wiener Process Concept

However carefully some of the pure mathematics underlying Wiener processes may have been explained, though, it’s probably the case that the whole matter still appears rather arid and abstract. Curious readers may be wondering just what a Wiener process looks like when it’s in action. To explain that, however, we have to go back in time and tell the story of the discovery of Brownian motion and the eventual intellectual conquest of the problems that it posed. Doing this will reveal what motivated Norbert Wiener to formulate the Weiner process concept and how he was able to do so.

People have been aware of Brownian motion for thousands of years, long before it was ever called by that name. Indeed, the Roman poet Lucretius, in an especially vivid passage from his De Rerum Natura (“On the Nature of Things”) described the motion of dust particles through the air in a way that, to modern ears, sounds strikingly accurate. Since Lucretius also believed in the existence of atoms (which, to him, were fundamental and indivisible particles of matter, not atoms as we know them today), he used his observations on Brownian motion to argue for their existence — a remarkable adumbration of precisely the use to which Albert Einstein would put his own much later work on the subject.

The next major chapter in our story takes place in 1827, when the Scottish botanist Robert Brown, in his efforts to study the pollen of the plant Clarkia pulchella, took some of that pollen and placed it in water so that he might better be able to examine it under a microscope. To Brown’s annoyance, the pollen, once suspended in water, just would not stay still. By repeating the experiment with some tiny particles of inorganic matter, Brown was able to strike down the hypothesis that whatever caused the particles’ jittery behavior had something to do with life itself. Nevertheless, no one knew what did cause it, and all were utterly at a loss to try and model it.

Still, Robert Brown had called the attention of the scientific community to this curious little anomaly and christened it with his name — Brownian motion.

Some decades later, Albert Einstein burst onto the scene and made the first truly serious attempt to unravel this mystery. Einstein devoted one of his three epochal 1905 papers to the subject of Brownian motion. In this paper, his central insight was one that he took from the subject of statistical mechanics.

In classical mechanics, most strongly associated with Isaac Newton and his work, objects and their physical dynamics are described individually. Properties like mass, charge, force, momentum, and so on are ascribed to objects individually and are given certain values. Then, with the help of the equations and formulas of classical mechanics, the precise physical behavior of every object can be charted once we know the initial values of the objects’ relevant properties.

This all sounds fine, but the approach encounters insurmountable problems whenever it must deal with systems containing large — or even not-so-large — numbers of objects. In a word, the complexity of the equations quickly grows to unfathomable levels. Indeed, as the three-body problem shows, even a relatively simple task like the description of the motion of three celestial bodies using Newton’s laws of motion and gravitation, given some initial conditions, is beyond the reach of classical mechanics. And if it cannot precisely chart the motion of three bodies, how quixotic is the task of using it to map out how countless billions of trillions of gas particles behave?

Therefore, when the architects of statistical mechanics turned their attention to explaining the behavior of gaseous particles — bumping and jostling with one another in gargantuan numbers as such particles so often do — they realized that the approach of classical mechanics was vain. And so, they chose to reorient the problem. Rather than explain the behavior of each gas particle — an impossibly difficult task — they decided to use statistics to describe the aggregate, group-level properties of these particles instead.

Einstein, therefore, realized that the key to an accurate mathematical description of Brownian motion lay in statistics and probability theory. He theorized that Robert Brown’s pollen behaved in the odd way that it did because moving pollen particles were constantly colliding with moving water molecules. These collisions produced the observed movements. The intricacies of Einstein’s theory on this matter — a theory which was experimentally confirmed in 1908 — need not worry us here. The primary thing to understand is that it was a statistical theory.

Norbert Wiener set to work refining and formalizing the mathematics behind the theory of Brownian motion yet further. As the behavior of pollen in water is random, Wiener understood that whatever other sort of process Brownian motion was, it must be a stochastic process. He further saw that this randomness meant that the distribution of pollen particles in water at any given time in the past could be of no help in predicting their positions in the future — leading him to draw on the mathematics of Levy processes. Lastly, he perceived that since Brownian motion was a naturally occurring process, the mathematics of the Gaussian normal distribution could be of great help in understanding it. Given that normal distributions appear virtually everywhere in nature — from the life spans of various animals to the distributions of things like height, IQ, memory, and reading ability among humans — it shouldn’t occasion a surprise if a normal distribution lies behind the veil of Brownian motion as well.

With this, all of the fundamental building blocks of the Wiener process concept were in place. Wiener just brought them all together. But his attempts to precisely explain an obscure natural curiosity turned out to have astonishing practical implications for things that went far beyond some water and some pollen.

Uses and Applications of the Wiener Process Concept

Wiener processes are used in everything from electronics engineering to derivatives trading. Some financial models outright assume that price movements in markets are Brownian, making Wiener processes the natural means by which to examine them and plan a trading strategy. Random walk models of markets also implicitly rely on Wiener processes, as random walks can be made to approximate a Wiener process.

A look at the code of this Python script also shows that computers seeking to model Brownian motion for any purpose must rely on random walks to do so. This is because computer logic operates discretely, and can thus only approximate the true continuity of a Wiener process.

Since electrons move around semiconductors according to a Wiener process, it would not be possible to build efficient — or even functional — semiconductors without them.

Real-World Examples of Wiener Processes

Since Wiener processes are just anything that behaves in the same way that Brownian motion does, there are real-world examples of Wiener processes occurring anywhere where Brownian motion occurs. These include:

• the original kind of Brownian motion that Robert Brown observed when looking at his pollen
• the movement of water molecules themselves
• how dust or pollutants diffuse through the air
• the movement of electrons (or holes) between the valence and conduction bands of a semiconductor
• the behavior of a stock, bond, forex, commodity, or other financial markets — at least according to some models
• the diffusion of calcium through the bones
• the movement of bits of plasma in cells

Up Next…

• The Complete Guide To Brownian Motion: Delve deeper into Brownian Motion!
• Meet Norbert Wiener – Complete Biography, History and Inventions: Lean more about the genius, Norbert Wiener, and his many contributions to science and mathematics.
• The Complete Guide to Cybernetics: Wiener first coined the term, cybernetics. Read our guide to learn more.