The first phase of the testing programme has been successfully
completed. In studying the results generated in Phase 1 it is important to note the
- The results generated and comments made only refer to the software actually used in the
trials. This should not simply be taken to mean the product name but also the release
number and version number of the software.
- The Phase 1 results are not intended to represent mesh independent solutions. They are
intended to represent converged solutions on "reasonable" meshes. In each test
case, the same computational mesh is used by each software product. Phase 2 simulations
can be used to explore simulations performed using finer meshes.
- The Phase 1 results do not make use of the most sophisticated physics available in each
of the software products. A base line set of characteristics has been set that allow a
fair comparison between the codes. Where model predictions are compared with experimental
data, these predictions can be improved through the use of more sophisticated physical
sub-models. Phase 2 simulations can be used to explore the benefits of using more
- The series of trials undertaken in this project should not be considered to be
definitive. They have been selected as a basis for exploring the potential of the
benchmarking process. It is intended that additional tests should be added to the suite of
In studying the outcome of the Phase 1 test cases, it is clear that
when identical physics is activated, identical computational meshes used and similar
convergence criteria applied, all of the software products tested are capable of
generating similar results. This is an important observation and suggests that
within the limitations of the tests undertaken that these three codes have a
similar basic capability and are capable of achieving a similar basic standard.
The one area that showed relatively poor agreement with theoretical
results concerned the radiation model performance. The six-flux radiation model while
capable of representing the average trends within the compartment, does not produce an
accurate representation of local conditions. It is clear from these results that users
should be aware of the limitations of the six-flux model when performing fire simulations.
Situations that are strongly radiation driven, such as the prediction of flame spread over
solid surfaces and structural response to fire should be treated with care. When using the
six-flux model, it is possible that target fuel surfaces would not be preheated by
radiation to the extent that would otherwise occur, thereby slowing the flame spread
The results from the CFD test cases are consistent with the view that
the basic underlying physics implemented within the codes are similar and provide a good
representation of reality. This should come as no surprise as all three software products
purport to model fluid dynamics processes using similar techniques. However, from a
regulatory viewpoint, it is reassuring to have an independent verification of this
similarity. In addition, where experimental results or theoretical solutions are
available, the software products have produced reasonable agreement with these results. No
doubt, it could be argued that improved agreement could be achieved if the spatial mesh
and time stepping are improved. This may be demonstrated in the Phase 2 simulations.
The results from the fire cases support the conclusions drawn from the
CFD test cases. While there are minor differences between the results produced by each of
the software products; on the whole they produce for practical engineering
considerations identical results.
A significant and somewhat reassuring - conclusion to draw from
these results is that an engineer using the basic capabilities of any of the three
software products tested would be likely to draw the same conclusions from the results
generated irrespective of which product was used. From a regulators view, this is an
important result as it suggests that the quality of the predictions produced are likely to
be independent of the tool used at least in situations where the basic capabilities
of the software are used.
A second significant conclusion is that within the limits of the test
cases examined and taking into consideration experimental inconsistencies and errors, all
three software products are capable of producing reasonable engineering approximations to
the experimental data, both for the simple CFD and fire cases.
What remains to be completed at this stage are the Phase 2 results
produced by the other testers. In Phase 2, the modellers are free to select which of the
test cases to repeat using the full capability of their software. These results will then
be checked by FSEG for their veracity.
Finally, the concept of the Phase 1 testing protocols has been shown to
be a valuable tool in providing a verifiable method of benchmarking and gauging the basic
capabilities of CFD based fire models on a level playing field. To further improve the
capabilities of the approach, it is recommended that additional test cases in the two
categories be developed and several of the fire cases be refined.