Publications Details

Quantitative results of the HRA empirical study and the role of quantitative data in benchmarking

Dang, Vinh N.; Massaiu, Salvatore; Bye, Andreas; Forester, John A.

In the International HRA Empirical Study, diverse Human Reliability Analysis (HRA) methods are assessed based on data from a dedicated simulator study, which examined the performance of licensed crews in nuclear power plant emergency scenarios. The HRA method assessments involve comparing the predictions obtained with the method with empirical reference data, in quantitative as well as qualitative terms. This paper discusses the assessment approach and criteria, the quantitative reference data, and the comparisons that use these data. Consistent with the expectations at the outset of the study, the statistical limitations of the data are a key issue. These limitations preclude concentrating solely on the failure counts defined by the Human Failure Event (HFE) success criteria and the failure probabilities based on these counts. In assessing quantitative predictive power, this study additionally uses a reference HFE difficulty (qualitative failure likelihood) ranking that accounts for qualitative observations in addition to the failure counts. Overall, the method assessment prioritizes qualitative comparisons, using the rich set of data collected on performance issues. Here, the quantitative predictions and data are used to determine the essential qualitative comparisons, demonstrating how quantitative and qualitative comparisons and criteria can be usefully combined in HRA method assessment.