The aim of the project reported here was to better understand the level of accuracy of three building energy simulation (BES) engines (‘engines’) — EnergyPlus™, DOE-2.1e, and DOE-2.2 — by identifying and investigating significant deviations between the performance predicted by these engines and actual performance as measured in the FLEXLAB® test facility at Lawrence Berkeley National Laboratory (LBNL). The specific test conditions included some of those prescribed in ANSI/ASHRAE Standard 140 - Standard Method of Test for the Evaluation of Building Energy Analysis Computer Programs. Detailed measurements of FLEXLAB performance, including indoor temperatures and heat fluxes and air-flow and water flow rates and temperatures in the Heating, Ventilating and Air Conditioning (HVAC) system, together with hourly weather data, were recorded and used in analyzing the simulation results from EnergyPlus v8.8, DOE-2.2 v3.65 and DOE-2.1e v127. These engines are commonly used in the United States for building energy code compliance, federal, state, and utility incentives programs, as well as energy efficient design of new buildings and energy retrofit of existing buildings.
Seven conventional overhead mixing ventilation scenarios were tested and each engine was found to have a similar level of agreement with the measurements of space-level heating and sensible cooling loads. These results provide useful information regarding the accuracy of these engines in predicting the cooling and heating load elements of whole building energy performance. This information is intended for practitioners who are concerned about transitioning between simulation tools with different engines and for managers of utility programs leveraging these tools for evaluating and/or projecting measure savings to be incentivized under their programs.
The results of the comparisons of simulated and measured performance indicate that the predictions from all three engines are not significantly different. The 24-hour average value of the absolute mean bias indicates the likely magnitude of the error in any particular case. The average mean bias is reduced by cancelation of overprediction in one case by underprediction in another. The daytime absolute mean biases, which may be more important for both energy performance and occupant comfort, are ~6%, presumably because of the greater complexity involved in simulating in the presence of solar radiation.
EnergyPlus typically overpredicts the cooling load and/or underpredicts the heating load by ~1.5% and the DOE-2 engines typically underpredict the cooling load by approximately the same amount. The Root Mean Square Error is relatively more sensitive to shorter term variations in the difference between predicted and measured loads; the three engines have similar values, ~10%, suggesting that the uncertainties in their predictions of peak loads may also be similar in magnitude. The implication of these results is that users, both designers and program analysts, can use EnergyPlus, DOE-2.1e, or DOE-2.2 to model conventional commercial buildings equipped with overhead mixing ventilation with a similar level of confidence.
Further work is required to better understand the variability in the level of agreement between the engine predictions and FLEXLAB measurements, where a particular engine will agree well with FLEXLAB in some cases and not so well in others and another engine will agree or disagree in different cases. As the sources of this variability are identified and eliminated or reduced significantly, it is recommended that the experimental capabilities and methods developed in the study reported here should be applied to validating heating and cooling load calculations for spaces with different types of furniture and miscellaneous loads. These methods should then be applied to low energy space conditioning systems in EnergyPlus including, in particular, radiant slab and radiant ceiling panel cooling and heating systems and ‘mixed mode’ systems that combine mechanical cooling and natural ventilation systems, focusing on controls, including control of thermal mass.
The work reported here addresses the conventional method of heating and cooling occupied spaces; other methods, such as the use of radiant heating and cooling systems have the potential to provide equivalent occupant comfort, or better, with lower energy consumption. These systems are addressed more explicitly in EnergyPlus but there is a need for empirical validation to give users the same level of confidence in modeling these systems that they have, or should have, in modeling conventional systems, based on the results presented here.