Why Simulation Validation Matters: Lessons from a Cooling Cup of Coffee 

Simulation offers engineers two major advantages: faster exploration of concepts and designs, and deeper understanding of the physics driving a product or process. 

Once a suitable model is established, engineers can change geometry, materials, or operating conditions and compare alternatives far faster than manufacturing and testing a prototype for each variation. This makes it practical to explore more concepts, identify promising designs earlier, and focus experimental effort where it adds the most value. 

However, every model is based on assumptions. Geometry is simplified, material properties are estimated, boundary conditions are idealized, and physical processes considered less important may be omitted. A model that includes every possible physical effect would often be too complex, slow, or expensive to solve. The challenge is determining which simplifications are acceptable. 

That is where experiments become essential. 

On the other hand, experiments also contain uncertainties. Sensors have limited accuracy, test conditions vary, data-processing methods influence the reported values, and one may not fully understand every observed effect.  

Simulation helps explain why a system behaves as it does. It provides access to temperature distributions, flow patterns, and other quantities that may be difficult to measure directly. By isolating mechanisms and examining their contributions, engineers can identify what controls performance, and which changes are most likely to improve it. 

A familiar example — how a cup of hot coffee cools — illustrates how combining simulation and experiment builds both physical understanding and confidence in predictions. 

A simple question with complex physics 

Coffee cooling appears straightforward, but several coupled processes may contribute: 

• heat conduction through the cup 

• convection between the cup and surrounding air 

• convection within the coffee 

• radiation to the surroundings 

• evaporation from the surface 

• water-vapor transport around the cup 

This is the fundamental modelling decision: which physics should be included, and which can reasonably be omitted? Simulation alone cannot always answer that question. A result may appear realistic even when an important mechanism is missing. Conversely, an unexpected measurement does not necessarily mean that the simulation is wrong. The measurement process may contain an unrecognised bias, an uncontrolled condition, or an incorrectly interpreted signal. 

Starting with a simple model 

A good modelling strategy is to begin with the simplest model that could reasonably represent the system and introduce complexity sequentially, when the evidence shows that it is necessary. 

For the coffee-cooling study, the initial model includes three familiar heat-transfer mechanisms: 

1. Conduction through the cup walls 

2. Convection inside the coffee and ambient air 

3. Thermal radiation to the surroundings 

The model produces a physically plausible cooling rate curve. Viewed in isolation, this result could easily be accepted, but a realistic-looking result is not necessarily an accurate one. 

Measuring the real cooling process

To validate the model, coffee was heated to approximately 80°C and poured into a cup at room temperature, afterwards its surface temperature was monitored over time using an infrared camera.

From the comparison one can see a clear discrepancy: the measured coffee cools significantly faster than the initial model predicts.

This disagreement indicates one of four possibilities:

1. an important physical process is missing

2. one or more model inputs are inaccurate

3. measurement contains an instrument bias

4. the simulated and measured quantities are not directly comparable

Each possibility should be investigated systematically.

Experiments reveal missing physics

The initial model included conduction, convection, and radiation but omitted evaporation.

When water evaporates from the coffee surface, it carries latent heat away from the liquid.

Because the latent heat of water is large, even a small amount of evaporation can remove a meaningful amount of energy.

After adding evaporative cooling to the model, the predicted cooling rate curve has a similar trend as the measurement. The measurement had therefore done more than identify an error: it provided insight into the physics driving the system.

However, adding evaporation does not produce perfect agreement. Further refinement is necessary.

Calibration addresses uncertain quantities

Simulation inputs are rarely known with complete certainty. In this model, potentially uncertain quantities included:

• initial temperatures

• ambient humidity and temperature

• material properties

• heat transfer coefficients

• evaporation rate

Experimental data can be used to calibrate parameters that cannot be measured accurately.

This must be done carefully. If too many parameters are adjusted simultaneously, different combinations may reproduce the same cooling curve. The model may then fit the experiment without correctly representing the underlying physics.

Figure 8. Liquid domain average of coffee cooling rate compared to IR measurements after adjusting initial temperatures, relative humidity and evaporation rate.

Experiments also contain assumptions

Every sensor and test method introduces uncertainty. An infrared camera measures infrared radiative flux reaching the detector and estimates the temperature using a calibrated model. That estimate depends on quantities such as:

• surface emissivity

• reflections

• camera focus and distance

• atmospheric transmission

For a grey body, emitted radiative heat flux can be represented by

$q_{rad}=\varepsilon \sigma T^4$,

where $\varepsilon$ is emissivity, $\sigma$ is the Stefan–Boltzmann constant and T is the surface temperature. The camera effectively performs an inverse calculation together with calibration and reflected radiation corrections to estimate temperature from measured radiation.

If the assumed emissivity is incorrect, the reported temperature will also be incorrect. If that measurement is treated as unquestionable truth, the simulation could be overfitted to a measurement bias, leading to incorrect conclusions.

Compare like with like

A valid comparison requires the simulation and experiment to represent the same observable.

An infrared camera measures the radiative flux from surfaces and converts it into temperature. It does not directly measure the volume-averaged temperature of the coffee.

The comparison should therefore use:

• the same physical location or surface

• the same spatial averaging method

• consistent temperature definitions

• equivalent initial conditions

• the same time reference

Figure 9. Sketch of the importance of evaluating simulation data at same measurement point as in the experiment.

In this example, comparing the measured surface temperature with the simulated temperature over the corresponding surface produces a better agreement.

This principle, of course, applies far beyond thermal imaging. Whether measurements come from a thermocouple or an accelerometer, the corresponding simulation result must be extracted in a comparable way.

Simulations improve experiments

Simulations can also help engineers understand and improve experiments.

An unexpected experimental result may be repeatable but still poorly understood. Simulations can isolate individual physical mechanisms, test competing hypotheses, and identify which effects could plausibly produce the observation. This can help determine:

• which quantities should be measured

• where additional sensors should be placed

• whether an observed trend is physically plausible

• whether the measurement method itself may be influencing the result

• which additional experiments would distinguish between competing explanations

For example, separate simulations could examine the sensitivity of the coffee measurement to emissivity, initial cup temperature, airflow, humidity, evaporation rate, or table contact. This does not prove which explanation is correct, but it helps prioritise the next experiment.

The relationship between simulations and experiments is a two-way exchange:

Experiments challenge and refine simulations, while simulations help interpret and improve experiments.

A measurement-driven modelling workflow

The coffee-cooling example illustrates a repeatable workflow for developing simulation models:

Figure 12. Diagram of the workflow used in the modelling process.

Part of the final validation step, the calibrated model should also be validated against a new experiment with different initial temperatures, coffee volume, cup geometry, or ambient conditions.

Numerical verification should also accompany this workflow. Mesh refinement, time-step studies, and solver checks establish that the model equations are being solved consistently.

Verification asks whether the equations are solved right; validation asks whether the right equations are solved.

What does this workflow provide?

A structured simulation–experiment workflow delivers several benefits:

1. It tests assumptions and verifies observations. Assumptions can be evaluated against observable behaviour, while simulations can isolate physical mechanisms, test hypotheses, and guide additional measurements.

2. It assesses repeatability and uncertainty. Repeated experiments and sensitivity studies distinguish robust trends from test variability.

3. It calibrates uncertainty systematically. Unknown parameters can be estimated using evidence rather than arbitrary values or trial and error.

4. It prevents false confidence. Detailed geometry, fine meshes, and attractive contour plots do not guarantee physical accuracy.

5. It improves traceability. A documented workflow records why the model changed and how each change affected agreement.

6. It quantifies accuracy. Error metrics and uncertainty bounds make model quality measurable rather than subjective.

Figure 13. Value of structured simulation-experiment workflow

From plausible results to faster, better-informed decisions

Experiments and simulations provide complementary information. Experiments capture how a physical system behaves under tested conditions. Simulations help explain the mechanisms behind that behaviour and enable rapid exploration of alternative designs and operating conditions.

Neither is unquestionable truth. Simulation predictions depend on assumptions and input data, while measurements depend on sensor performance, experimental control, and interpretation. When they are combined, discrepancies become opportunities to learn.

The coffee cooling study demonstrates this process. Adding evaporation improved the predicted cooling behaviour, calibration addressed uncertain inputs, and comparing equivalent quantities made the assessment more meaningful. The result was not simply a better-fitting curve, but a clearer understanding of the processes responsible for cooling.

Validation helps make simulation’s speed and insight useful for engineering decisions. With confidence established for the intended application, engineers can explore alternatives quickly, focus on the mechanisms that matter, and target further experiments more effectively.

Ultimately, the goal is not to force a simulation to reproduce one experiment. It is to establish when, where, and with what uncertainty the model can support predictions, so that faster design exploration is backed by sound physical understanding.

Figure 14. Value of experiments in simulations

References

[1] Mayer, Jurgen; Khairy, Khaled; Howard, Jonathon (May 12, 2010). “Drawing an elephant with four complex parameters”. American Journal of Physics. 78 (6): 648–649.