EXECUTIVE SUMMARY

The purpose of the proposed standards/benchmarks is to aid the fire safety approvals authority in assessing the appropriateness of using a particular model for a particular fire modelling application. This benchmark has been split into two phases. The first phase is intended to test all the software products using identical or equivalent models. The second phase of testing allows the full range of the software’s capability to be demonstrated. In each phase, five non-fire (CFD) and five fire cases are tested.

The first phase of the testing programme has been successfully completed. In studying the outcome of the Phase 1 test cases, it is clear that when identical physics is activated, identical computational meshes used and similar convergence criteria applied, all of the software products (PHOENICS, CFX and SMARTFIRE) tested are capable of generating similar results. This is an important observation and suggests – within the limitations of the tests undertaken – that these three codes have a similar basic capability and are capable of achieving a similar basic standard. While there are minor differences between the results generated by each of the software products; on the whole they produce – for practical engineering considerations – identical results. From a regulatory viewpoint, it is reassuring to have an independent verification of this similarity.

The one area that showed relatively poor agreement between model predictions and theoretical results concerned the six-flux radiation model performance. The six-flux radiation model while capable of representing the average trends within the compartment, does not produce an accurate representation of local conditions.

CFX, PHOENICS and SMARTFIRE all provide alternative radiation models which may offer superior performance. This has been demonstrated for the CFX 12-ray Shah-Lockwood model within this document. It should be noted that the six-flux model was used as it was common to both PHOENICS and SMARTFIRE, and CFX could be made to crudely approximate the six-flux model. However, CFX does not possess a six-flux model and so the Shah-Lockwood model was used with a single ray to give the closest approximation possible to the six-flux model. It should be noted that the developers of CFX generally advises that the CFX radiation model should never be used with a single ray. As mentioned previously the intention of phase-1 was to test the codes in as similar a manner as possible to try and give an unbiased reflection of how the codes compared. This task would not have been possible unless the CFX single ray radiation model was used.

A significant – and somewhat reassuring - conclusion to draw from these results is that an engineer using the basic capabilities of any of the three software products tested would be likely to draw the same conclusions from the results generated irrespective of which product was used. From a regulators view, this is an important result as it suggests that the quality of the predictions produced are likely to be independent of the tool used – at least in situations where the basic capabilities of the software are used.

A second significant conclusion is that within the limits of the test cases examined and taking into consideration experimental inconsistencies and errors, all three software products are capable of producing reasonable engineering approximations to the experimental data, both for the simple Computational Fluid Dynamics (CFD) cases (i.e. non-fire cases) and full fire cases.

An important element of this work concerned the procedures for undertaking the testing. While all of the test cases using all of the codes were run by a single organisation – in this case the Fire Safety Engineering Group (FSEG) at the University of Greenwich – the code developers also were requested to run an independent selection of the test cases as specified. This was necessary to verify that the results produced in this report are a true and fair representation of the capabilities of the various software products under the specified test conditions. This has proven to be quite useful as it brings the developers into the benchmarking process and it eliminates issues concerning fairness and biased reporting of results.

What remains to be completed at this stage are the Phase 2 results produced by the other testers. In Phase 2, the modellers are free to select which of the test cases to repeat using the full capability of their software to give the best possible representation of the case. These results will then be checked by FSEG for their veracity.

Finally, the concept of the Phase 1 testing protocols has been shown to be a valuable tool in providing a verifiable method of benchmarking and gauging the basic capabilities of CFD based fire models on a level playing field. To further improve the capabilities of the approach, it is recommended that additional test cases in the two categories (basic CFD non-fire and fire) be developed.