Skip to the content of the web site.

Inverse Analaysis

What is an inverse problem?

Much research in science and engineering is devoted to constructing numerical models of physical systems. Given a complete description of the physical system, these models can be used to predict how the system affects its surroundings. This is called the direct or forward problem. When some aspects of physical system are unknown, however, it is sometimes possible to infer these characteristics from a known system response; this is the inverse problem.

The relationship between the forward and inverse problem can be thought of in terms of cause and effect, where the cause may include the system boundary conditions; initial conditions; thermophysical properties; internal heat sources; and the problem geometry, and the effect is an observed or measured system response. Specifying this causal relationship and predicting its effect is the forward problem, while in the inverse problem the causal relationship is inferred from the observed effect.

Mathematically ill-posed problems

Inverse problems are challenging to solve because they are ill-posed. Ill-posed problems violate one or more of the criteria for well-posed problems, originally defined by Jacques Hadamard in 19231

  1. The problem must have a solution;
  2. The solution must be unique, and;
  3. The solution must be stable under small changes to the data.

At first, mathematicians thought problems that violated these criteria made no practical or physical sense, and that there was no need to attempt solutions to these problems. It has subsequently been shown, however, that mathematically ill-posed inverse problems are ubiquitous throughout science and engineering.

Inverse problems can be categorized as either parameter estimation or design optimization problems. These two classes differ only by the data used to infer the system characteristics: in parameter estimation problems, the system characteristics are inferred from experimental data, while the objective of design optimization is to infer the optimal design from a specified optimal design outcome.

Parameter estimation

Parameter estimation problems usually satisfy the first criterion, since something is responsible for the observed system response. Instead, they violate the third criterion and "almost" violate the second criterion because many different system configurations exist that produce a very similar observed system response. If different system configurations produce similar observed data, the corollary is that the recovered solution will be highly sensitive to small perturbations in the data, which are inevitable in an experiment.

A classic (and silly) example2 is the problem of a knight trying to guess what kind of dragon he is hunting from footprints he finds in the forest. This is a difficult task if many types of dragon produce similar footprints, and becomes impossible if the footprints are even slightly smeared in mud.

Knowing the footprint that a particular species of dragon leaves behind is easy (forward problem), but inferring the species of dragon from the footprint (inverse problem) is harder, and violates Hadamard's third criterion for well-posed problems.
dragon footprints

Design optimization

Unlike parameter estimation, design optimization problems often violate Hadamard’s first criterion since an optimal design outcome may be specified that cannot possibly be produced by the system. On the other hand, the existence of multiple designs that produce the optimal outcome constitutes a violation of the second criterion. In an everyday example, say you go to the store to buy concentrated orange juice. Hadamard's second criterion would be violated if the store has two brands of orange juice, both of which taste good and cost the same. If the store were sold out of orange juice, Hadamard's first criterion would be violated.

Ill-posed problems at the grocery store.
orange juice

For a more industrial example, consider the design of a drying furnace in which the goal is to find heat flux distribution over the top surface that produces a specified heat flux over the product surface on the bottom3. Both continuously-varying and discrete heat flux distributions shown below produce the same heat flux distribution over the product, which illustrates a violation of Hadamard's second-criterion.

A design optimization problem that violates Hadamard's second-criterion: both (a) discrete and (b) continuous heat flux distributions over the heater surface produce nearly identical heat flux distributions over the product surface3.
enclosure design problem

The role of prior knowledge

Inverse problems are mathematically ill-posed due to an information deficit. In the parameter estimation case (including TiRe-LII nanoparticle sizing and axisymmetric tomography), the measurements barely provide sufficient information to specify a unique solution, and in some cases (e.g. 2D chemical species tomography) the data could be explained by an infinite set of candidate solutions. Accordingly, inverse analysis resolves this ambiguity by injecting additional information into the problem. Often, this information is based on expectations of the solution; for example, an expected range of nanoparticle sizes, a degree of spatial smoothness in a species concentration distribution, or additional desired attributes of an optimal design. A classic example is the "backwards reasoning" used by Sherlock Holmes.

Sherlock and Mycroft Holmes make inferences by combining observations and prior knowledge.

In this example, the observations by themselves (“sunbaked skin”, “ammunition boots”, "large stature," "wearing black clothes", etc.) tell Sherlock and Mycroft very little about the situation, but a complete picture is obtained when these observations are combined with prior knowledge (British soldiers of that era served in Afghanistan, black clothes denoted mourning in Victorian society, etc.)

When combining prior knowledge with measurements, however, care must be taken to ensure that the information obtained from these two sources is weighted appropriately. If not enough emphasis is placed on the prior knowledge, the solution will be ambiguous. On the other hand, if too much emphasis is placed on the prior knowledge, the inferred solution will correspond exactly with expectations; one could think of this as "the Law of Self-Fulfilling Prophecies."

The use of salt in the preparation of chicken soup is a good analogy to the role prior information plays in inverse analysis. Salt is added to "bring out the flavor" in the broth, just as prior knowledge should be used to enhance the information content of the data. Add too much salt, however, and your soup tastes like salt!

  1. J. Hadamard, 1923, Lectures on Cauchy's Problem in Linear Partial Differential Equations, Yale University Press, New Haven CT.
  2. C. F. Bohren, D. R. Huffman, 1983, Absorption and Scattering of Light by Small Particles, John Wiley and Sons, New York NY.
  3. F. França, J. R. Howell, O. A. Ezekoye, 2001, "Inverse boundary design containing radiation and convection heat transfer," Journal of Heat Transfer, 123, pp. 884-891.