CONCLUDING COMMENTS

The first phase of the testing programme has been successfully completed. In studying the results generated in Phase 1 it is important to note the following points:

1. The results generated and comments made only refer to the software actually used in the trials. This should not simply be taken to mean the product name but also the release number and version number of the software.
2. The Phase 1 results are not intended to represent mesh independent solutions. They are intended to represent converged solutions on "reasonable" meshes. In each test case, the same computational mesh is used by each software product. Phase 2 simulations can be used to explore simulations performed using finer meshes.
3. The Phase 1 results do not make use of the most sophisticated physics available in each of the software products. A base line set of characteristics has been set that allow a fair comparison between the codes. Where model predictions are compared with experimental data, these predictions can be improved through the use of more sophisticated physical sub-models. Phase 2 simulations can be used to explore the benefits of using more sophisticated physics.
4. The series of trials undertaken in this project should not be considered to be definitive. They have been selected as a basis for exploring the potential of the benchmarking process. It is intended that additional tests should be added to the suite of test cases.

In studying the outcome of the Phase 1 test cases, it is clear that when identical physics is activated, identical computational meshes used and similar convergence criteria applied, all of the software products tested are capable of generating similar results. This is an important observation and suggests – that within the limitations of the tests undertaken – that these three codes have a similar basic capability and are capable of achieving a similar basic standard.

The one area that showed relatively poor agreement with theoretical results concerned the radiation model performance. The six-flux radiation model while capable of representing the average trends within the compartment, does not produce an accurate representation of local conditions. It is clear from these results that users should be aware of the limitations of the six-flux model when performing fire simulations. Situations that are strongly radiation driven, such as the prediction of flame spread over solid surfaces and structural response to fire should be treated with care. When using the six-flux model, it is possible that target fuel surfaces would not be preheated by radiation to the extent that would otherwise occur, thereby slowing the flame spread process.

The results from the CFD test cases are consistent with the view that the basic underlying physics implemented within the codes are similar and provide a good representation of reality. This should come as no surprise as all three software products purport to model fluid dynamics processes using similar techniques. However, from a regulatory viewpoint, it is reassuring to have an independent verification of this similarity. In addition, where experimental results or theoretical solutions are available, the software products have produced reasonable agreement with these results. No doubt, it could be argued that improved agreement could be achieved if the spatial mesh and time stepping are improved. This may be demonstrated in the Phase 2 simulations.

The results from the fire cases support the conclusions drawn from the CFD test cases. While there are minor differences between the results produced by each of the software products; on the whole they produce – for practical engineering considerations – identical results.

A significant – and somewhat reassuring - conclusion to draw from these results is that an engineer using the basic capabilities of any of the three software products tested would be likely to draw the same conclusions from the results generated irrespective of which product was used. From a regulators view, this is an important result as it suggests that the quality of the predictions produced are likely to be independent of the tool used – at least in situations where the basic capabilities of the software are used.

A second significant conclusion is that within the limits of the test cases examined and taking into consideration experimental inconsistencies and errors, all three software products are capable of producing reasonable engineering approximations to the experimental data, both for the simple CFD and fire cases.

What remains to be completed at this stage are the Phase 2 results produced by the other testers. In Phase 2, the modellers are free to select which of the test cases to repeat using the full capability of their software. These results will then be checked by FSEG for their veracity.

Finally, the concept of the Phase 1 testing protocols has been shown to be a valuable tool in providing a verifiable method of benchmarking and gauging the basic capabilities of CFD based fire models on a level playing field. To further improve the capabilities of the approach, it is recommended that additional test cases in the two categories be developed and several of the fire cases be refined.